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To the President, the Congress, the Secretary of State and the American People: 

Pursuant to Public Law 110-113, the United States Advisory Commission on Public Diplomacy (the 
"Commission") hereby submits a report on U.S. government public diplomacy policies and programs. 

The Commission is a bipartisan panel created by the Congress in 1948 to advise and make recommen- 
dations on the conduct of public diplomacy. During the past year, the Commission focused its efforts on the 
critically important need to effectively evaluate public diplomacy programs. Indeed, Under Secretary of State 
Judith McHale's public diplomacy strategic framework emphasizes the importance of evaluating and assess- 
ing the impact of public diplomacy. However, we are concerned that the Department's current efforts are not 
sufficiently rigorous, nor adequately coordinated, so as to effectively measure policy outcomes, including 
most importantly the real world impact of our programs on the perspectives and attitudes of foreign publics. 
Moreover, there does not appear to be any meaningful effort by the Department to incorporate quantitative 
methods in evaluating the effectiveness of public diplomacy programs. 

In an effort to leverage its limited resources, the Commission, with the assistance of Under Secretary 
McHale, entered into a cooperative agreement with the Lyndon B. Johnson (LBJ) School of Public Affairs 
of the University of Texas at Austin. Although other universities have established programs for the study of 
public diplomacy, the Commission believed that the LBJ School was well suited to undertake the project on 
the basis of its work related to the quantitative measurement of public policy programs. At the direction of the 
Commission, over the course of an academic year a graduate policy workshop at the LBJ School conducted 
research and developed a notional quantitative model (PD-MAP) for evaluating such programs. 

As with all government programs, it is critically important to measure the effectiveness of public diplo- 
macy programs. Although the State Department is engaged in efforts to evaluate the effectiveness of certain 
public diplomacy programs, primarily through the Bureau of Educational and Cultural Affairs (ECA) and the 
Office of Policy, Planning and Resources (R/PPR), the Department does not regularly employ replicable 
quantitative methods capable of measuring the extent to which we are achieving the desired foreign policy 
outcomes on the basis of our public diplomacy efforts. Significantly, the Department's Mission Activity Tracker 
(MAT), a tool developed in 2007 at the behest of the Office of Management and Budget, the Government 
Accountability Office, and the Department's Office of Inspector General, evaluates public diplomacy outputs 
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but not public diplomacy outcomes. For example, the MAT captures the number of people who attend a 
public policy lecture held at a U.S. Embassy (an output), but it does not attempt to evaluate whether or not 
the attendees' understanding of, or attitude with respect to, U.S. foreign policy was influenced favorably by 
the program (an outcome). 

The Commission hereby accepts the report presented by the LBJ School, commends the faculty and 
student work contained therein, and forwards its recommendations to the Department. Although there may 
well be limitations inherent in an academic model that make it difficult to apply comprehensively in evaluating 
a wide range of public diplomacy programs, we encourage the Department to consider incorporating quanti- 
tative methods into its ongoing evaluation work. Moreover, we believe that the notional model may have value 
in strategic planning and resource allocation decisions faced by senior officials. 

In accepting the LBJ School report, the Commission acknowledges the important contributions of those 
public diplomacy practitioners working in the Department and in other government agencies, both here and 
abroad. We thank these public servants for their ongoing efforts, and their cooperation in connection with this 
report. We hope that this report might provoke thoughtful consideration of the value of quantitative evaluation 
methods in the context of public diplomacy. 

Finally, we offer an observation about the Commission and its work. We fully appreciate that the Com- 
mission is charged by the President and the Congress to undertake important advisory and oversight work in 
connection with public diplomacy; this work has been made all the more important following the 9/1 1 attacks 
on our nation and the imperative of conducting thoughtful, effective public diplomacy in support of our policy 
objectives. We have been frustrated at times by limitations related to re-authorization, funding, staffing and 
access to Department officials. That said, we remain determined to fulfill our mandate through the timely 
examination and reporting on key public diplomacy issues, interaction with Washington and embassy officials, 
and engagement with universities, think tanks and the NGO community. We thank you for your interest. 



Respectfully Submitted, 




Penne Korth Peacock 
(Texas) 



(Colorado) 



(Texas) 




(Virginia) 




(Pennsylvania) 
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Executive Summary 



The information contained in this report represents the culmination of a year-long project 
conducted by a research team at the LB J School of Public Affairs at The University of 
Texas at Austin. As a result of their research, the team created a flexible, quantitative 
assessment tool for measuring the effectiveness of public diplomacy (PD) programs. This 
report follows this generalized structure: 

■ Overview of current evaluation efforts 

■ New measurement tool and how it fits in with current measurement tools 

■ Instructions on how to modify the measurement tool to fit Congress and DoS's 
needs 

■ Recommendations for future evaluation efforts 

Public Diplomacy Model for the Assessment of Performance (PD-MAP) 
What is it? 

PD-MAP is a flexible framework that allows an evaluator to quantify the results of public 
diplomacy programs and evaluate their success in meeting the following three strategic 
goals or outcomes of all public diplomacy programs: 

1 . Increasing understanding of US policy and culture 

2. Increasing favorable opinion towards the US 

3. Increasing the US's influence in the world 

What does it do? 

In the measurement process, an evaluator creates appropriate measurements for PD 
programs, sets the priority for each measurement and sub-outcome (such as increasing 
dissemination of information), and sets the risk that a department is willing to assume for 
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each outcome. Public diplomacy and public affair officers collect the data in the field and 
enter it into an interface such as the Mission Activity Tracker. Using PD-MAP, an 
evaluator can then run calculations based on the measurements and produce scores for 
PD programs based on their success in meeting their strategic goals. 

How should it be used? 

The scores produced by the PD-MAP should complement expert analysis to 

■ Identify what programs are achieving their strategic goals 

■ Identify what programs are underperforming 

■ Determine where and how resources should be allocated based on the Office of 
Public Diplomacy and Public Affair's mission 



For instance, if the strategic mission for Muslim countries is to increase the general 
population's understanding of US culture, the Office of PD and PA will use PD-MAP to 
identify what programs have shown success in reaching a large number of people and 
increasing their understanding of the US. The Office can then increase the scope of that 
particular program or assign additional staff to administer it. If a program is 
underperforming, the appropriate bureau or department can investigate, discover why it is 
failing, and work to improve the program or close it. 

Conclusions and Recommendations for Public Diplomacy Evaluation 

In their research, the team found the following problems with current evaluations efforts 
within the Office of PD and PA: 

■ No coordination between PD/PA departments 

■ Duplication of evaluation efforts 

■ No uniform scale or basis for analyzing or comparing different programs 

■ No single department coordinates or is held responsible for measurement 
standards 

■ Insufficient relationship between program planning and evaluation 
Based on their analysis, the research team makes the following recommendations: 

■ Empower one central evaluation department such as the Evaluation and 
Measurement Unit (EMU) to oversee evaluation for PD and PA departments and 
programs 

■ Roll out PD-MAP into EMU's current evaluation and data gathering efforts to 
create a single measurement and evaluation process 

■ Coordinate OMB's evaluation requirements with this single evaluation process 

■ EMU should run a pilot program to test the evaluation process 

■ Upon success, combine evaluation scores with expert analysis of the programs to 
identify successful and underperforming programs in quarterly or annual reports 
for Congress and DoS 
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■ Use the evaluations to plan or alter future PD programming 
Going Forward 

The version of PD-MAP presented in this report is the first step towards developing a 
comprehensive measurement tool. DoS and a second independent research team should 
work to include additional information from PD officers currently working in the field to 
extend and improve the sample measurements described within this report. A second 
team should visit embassies, learn how PD officers administer programs and gather 
information, and work with them to perfect the measurement tool. Involving actual users 
in developing the tool will improve the tool's capability as well as users' receptivity 
toward implementing the tool. 
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Chapter 1. Introduction 



Public diplomacy is an important though often over-looked tool for promoting 
democracy, security, and economic stability throughout the world. Since 9/11, the United 
States government has renewed its interest in public diplomacy as a soft power approach 
to mitigating threats to US security and countering the spread of disinformation. The US 
employs public diplomacy as tool for increasing regional and global stability, 
strengthening the US's global economic ties, and improving political relations for the 
benefit of the international community as well as the US. According to the American 
Academy of Diplomacy, the US Government attempts to "understand, inform, engage 
and influence global audiences, reaching beyond foreign governments to promote greater 
appreciation and understanding of US society, culture, institutions, values and policies." 1 
This broad definition of public diplomacy serves as a basis or beginning point for 
analyzing the purpose, design, and effectiveness of the US's PD efforts; however, we 
must expand this definition and clearly identify PD efforts, their targets, and their specific 
missions to determine which public diplomacy programs most effectively promote US 
security and economic stability. 

In August 2009, the US Advisory Commission on Public Diplomacy approached the 
Lyndon Baines Johnson School of Public Affairs at The University of Texas at Austin 
with the opportunity to develop a new method for assessing the effectiveness of public 
diplomacy efforts. The Commission requested that the LBJ School undertake a research 
project to review current public diplomacy measurement methods, assess gaps in the 
various measurement methods, and develop a comprehensive measurement framework. 
The purpose of this report is to iterate the research team's assessment of ongoing 
measurement methods and describe the quantitative measurement framework it 
developed to supplement and support those methods. 

To develop a quantifiable method for evaluating the effectiveness of public diplomacy 
efforts, the research team identified three main goals or desired outcomes of public 
diplomacy, developed measurements of the various programs' outputs, and created a 
flexible framework — the Public Diplomacy Model for the Assessment of Performance 
(PD-MAP) — to assess whether the outcomes meet DoS's goals. The purpose of this 
model is to provide a tool that DoS can use in coordination with other methods to 
understand the relationship between a PD program's goal, design, and implementation 
and its effectiveness. 



1 American Academy of Diplomacy, A Foreign Affairs Budget for the Future: Fixing the Crisis in Diplomatic Readiness (October, 
2008), p. 24. Online. Available: http://www.stimson.org/budgeting/Publications/Long_Final_ll_08.pdf. Accessed: 
September 21, 2009. 
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Report Organization 



This report consists of eight major sections. The first section introduces two main public 
diplomacy measurement methods, highlights various US government agencies' 
evaluations of US public diplomacy efforts, and introduces the PD-MAP approach. The 
second section gives an overview of how the research team developed its model. The 
third section describes the model. Sections four, five, and six explain in detail the three 
outcomes the team identified as the core goals of US public diplomacy. Section seven 
explains how the model can be implemented in the field, discusses the benefits and 
challenges of using this method, and provides the research team's recommendations for 
improving DoS's approach to evaluating its public diplomacy programs. 



Current Measurement Methods 

Public diplomacy is a soft power tool that aims to win the hearts and minds of its 
recipients. DoS worked for many years to measure the success or effectiveness of PD 
efforts to obtain this nebulous goal. In 2007, at the request of The Office of Management 
and Budget (OMB), the General Accountability Office (GAO), and the Office of 
Inspector General, DoS created the Mission Activity Tracker (MAT) to gather 
quantifiable data and use a performance measurement system to assess the effectiveness 
of US public diplomacy efforts in the field. 2 The Office of Policy, Planning and 
Resources (R/PPR) of the Under Secretary for Public Diplomacy and Public Affairs 
developed fifteen performance measures that missions across the globe could gather and 
report using the MAT. While the MAT has been successful in quantifying information 
regarding public diplomacy activities, it does not draw a correlation between the event 
itself and the desired outcomes. 

The Office of Policy, Planning, and Resources also created the Public Diplomacy Impact 
(PDI) Project with the goal of tying output data with performance measurements. This 
project focused on six main outcomes: 

• Initiation of positive change to local communities 

• Incorporation of US sponsored materials 

• Satisfaction scores of public diplomacy programs 

• Improved or increased understanding of US 

• Favorability rating of US 



2 U.S. Department of State, Public Diplomacy Evaluation Office, 

http://exchanges.state.gov/programevaluations/pmm/pdeo.html (accessed June 20, 2010). 
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Reduced anti- Americanism 



The PDI Project examined data from selected US missions and analyzed the correlation 
between their PD activities and the desired outcomes. This project represented the first 
quantitative attempt to find a relationship between activities and outcomes. While this 
information provides valuable qualitative input, it lacks standardization across posts. This 
makes it difficult for the Office of PD and PA to compare programs within a country or 
assess the progress of individual programs over time. 

Coordinating Evaluations 

Although several departments within DoS focus on public diplomacy, there is no single 
entity dedicated to evaluating public diplomacy. The different PD-related departments 
rarely interact, which causes a duplication of efforts and prohibits problem solving and 
transferring best practices across departments. Also, their different measurement 
approaches and scales make it difficult for DoS or other interested parties to compare and 
identify the most effective programs. 

Governmental agencies, independent of DoS, measured the efficacy of American public 
diplomacy programs and found them to be lacking in key areas such as population 
outreach, proper training, and PD officer recruitment. In 2003, the GAO began a 
comprehensive study of DoS's attempts to improve the American image; it has since 
issued three reports, all of which state that current efforts desperately need improvement. 
The most recent report, published in 2009, claims, "The United States' current national 
communication strategy lacks a number of desirable characteristics identified by GAO, 
such as a clear definition of the problem, desired results, and a delineation of agency 
roles and responsibilities." 3 Another GAO report, created in 2006, found that DoS's 
public diplomacy activities were inefficient in reaching the Muslim world. 4 The GAO 
clearly considers public diplomacy to be an important to national security tool — the 
office listed improving the US image abroad as fifth on its list of top thirteen priorities in 
the 2009 report. 5 

In 2008, OMB rated PD field operations as "not effective and not performing." 6 It 
assessed the whole of US PD as simply "adequate" in 2006, though exchanges received 
an "effective rating" 7 The Public Diplomacy Council, a nonprofit organization consisting 



3 US Government Accountability Office, "US PUBLIC DIPLOMACY: Key Issues for Congressional Oversight," 
http://www.gao.gov/new.items/d09679sp.pdf Accessed: April 12,2010. 

4US Government Accountability Office, "DoS Efforts to Engage Muslim Audiences Lack Certain Communication Elements and 
Face Significant Challenges" http://www.gao.gov/new.items/d06535.pdf Accessed: April 13,2010 

5 US Government Accountability Office, "US PUBLIC DIPLOMACY: Key Issues for Congressional Oversight," 
http://www.gao.gov/new.items/d09679sp.pdf Accessed: April 12,2010. 

6 US Office of Management and Budget, "Program Assessment: Public Diplomacy," 

http://www.whitehouse.gov/omb/expectmore/summary/10004600.2006.html Accessed:April 22,2010 

7 US Office of Management and Budget, "Performance Reports of the Federal Government." 
www.whitehouse.gov/omb/expectmore Accessed: April 22,2010. 
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of former diplomats from both DoS and USIA, agreed with both the GAO and the OMB 

o 

that the government is simply not doing enough in regards to its image overseas. 

In an effort to raise PD programming to a higher standard, OMB listed an annual 
efficiency performance measure, seven annual output performance measures, ten annual 
outcome performance measures, and three long-term outcome performance measures as 
obtainable measurement goals in its public diplomacy performance evaluation 
recommendation for 201 1. The measures vary from the number of people in a foreign 
audience who approve of the United States to the user satisfaction of people visiting 
embassy websites and the number of strategic partnerships leveraged. 9 These 
performance measures show not only the true scope and variance of the public diplomacy 
realm, but the importance of quantifying DoS's efforts. 



PD-MAP 



The research team from the LBJ School recognized that the difficulty in any performance 
measurement of public diplomacy is developing creative, replicable, and transferable 
ways to measure outcomes, not simply outputs — which are often employed in 
measurement approaches as described above. An output measures an activity produced 
by a PD effort, whereas an outcome measures the impact of a PD effort in relation to 
DoS's goals for public diplomacy — the key information DoS is seeking. An example of 
an output is the number of people who attended a DoS-sponsored cultural program. The 
outcome is the attendees' increased understanding of the US's cultural diversity. Current 
public diplomacy measurement focuses on outputs, which are easier to quantify and 
track; however, the PD-MAP measures outcomes, which gets to the heart of assessing the 
effectiveness of PD efforts. 

Evaluating the effect of a public diplomacy activity independent of outside factors is the 
greatest challenge in measuring outcomes. The point of the measurement process is to 
determine whether a PD effort had an effect on the target audience, but since PD efforts 
do not occur in a vacuum, it is not always clear whether the PD effort was the cause of 
the change or the change was a result of another factor. Some activities may be easier and 
clearer to measure than others. For example, a Public Diplomacy Officer (PD Officer) 
can gauge the knowledge of the audience before and after a DoS sponsored lecture 
through entrance and exit surveys, allowing the PD Officer to derive an immediate and 
direct measurement. However, in most cases, such as an education exchange program, the 
PD effort's impact will not be clear for years. The PD-MAP provides short-term and 
long-term measurements that can be tracked. Also, all departments can use the 
framework, thereby providing a common scale for decision makers to use to compare the 
effectiveness of all DoS public diplomacy programs. 



8 Walker Pinkus, "GAO Calls for A New Priority On Public Diplomacy," The Washington Post. 

http://www.washingtonpost.com/wp-dyn/content/article/2009/01/ll/AR2009011102122.html Accessed: April 12, 2010. 

9 US Office of Management and Budget, "Public Diplomacy Performance Measures (Updated for FY 2011)". 
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Chapter 2. Methodology 



The research team's first step in developing the PD-MAP was to pinpoint the core goals 
of US public diplomacy efforts. The research team gathered information on the goals 
through interviews and a focus group with experts, conducting a survey of embassy staff, 
reviewing DoS and embassy websites and conducting a literature review. The survey and 
focus group were particularly useful for not only gathering information on public 
diplomacy goals, but also learning how potential end-users might incorporate the PD- 
MAP into planning. Once the core goals were determined, the team reviewed its 
catalogue of various PD programs to understand how the programs worked to meet the 
goals. From this information, the team was able to create measurements for several PD 
activities. 

The project team employed the following methods for collecting information: 

• Review of current public diplomacy programs 

• Survey of public diplomacy professionals and academia. 

• Focus Group 

• Expert Speakers 

The reader should note that the measurements discussed in this report are not 
comprehensive. EMU should assess the suggested measurements for relevance and 
accuracy and include additional measurements where appropriate. The research team did 
not have access to program information, thus it relied on information posted by DoS and 
embassies on their web sites. Embassy's web sites vary greatly and do not necessarily 
contain full information about their activities. Additionally, the team did not have access 
to embassy staff through DoS, so the team performed the survey using its own network of 
embassy staff and academics who are either highly familiar with public diplomacy 
programs or who worked in embassies previously. While the information the team 
gathered was extremely useful for creating the model, it is limited due to resource and 
access constraints. 



Review of Current Public Diplomacy Programs 

The purpose of this research was to provide background information about current PD 
programs and practices. A comprehensive document outlining all PD programs or those 
only at individual embassies may or may not exist, but the team was able to pull general 
descriptions of programs from other sources, including DoS and the Office of PD and 
PA's websites and various embassy websites around the world. This section describes a 
limited number of public diplomacy programs on which the research team gathered 
information. 
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Public Diplomacy Programs 

There are five bureaus and offices that report to Under Secretary of State for Public 
Diplomacy and Public Affairs Judith McHale. They are: 

■ Bureau of Educational and Cultural Affairs (ECA) 

■ Bureau of International Information Programs (IIP) 

■ Bureau of Public Affairs (PA) 

■ Office of Policy, Planning and Resources (R/PPR) 

■ Office of Private Sector Outreach (R/PSO) 10 

Bureau of Educational and Cultural Affairs 

The ECA operates DoS's educational and cultural exchange programs, as well as 
leadership programs. In total, the ECA oversees 83 exchange programs. These exchanges 
cover the cultural, sports, academic and professional fields and sponsor citizens from 160 
countries. The purpose of these programs is to foster "mutual understanding between the 
people of the United States and the people of other countries to promote friendly, and 
peaceful relations, as mandated by the Mutual Educational and Cultural Exchange Act of 
1961. 1,11 

Bureau of International Information Programs 

The role of IIP is to convey information about US "policy, society, and values" to foreign 
audiences through a speaker program and various publications, which it translates into 
local languages to best reach its target audience. The IIP is also working to develop new 
communication tools, such as social networking sites through the Digital Outreach Team 
(DOT). The DOT visits Persian, Urdu, and Arabic language blogs, chat rooms, and 

12 "~ 

websites and engages with visitors. The IIP also operates America.gov, the US Speaker 
Program, American Corners, Information Resource Centers (IRCs), writes various 
publications, and trains Information Resource Officers to "deliver research, training, and 
cross-cultural programs to international audiences." 13 

Prior to the 1998 bombings of US embassies in Tanzania and Kenya, DoS operated 
facilities (American Centers) where local populations could learn about American 
culture, history, current events, and the US government through newspapers and other 
print publications, lectures, classes, films, and staff presentations. They were located in 



10 Kennon H. Nakamura and Matthew C. Weed, U.S. Public Diplomacy: Background and Current Issues, CRS Report for 
Congress (Congressional Research Service, 2009), p. 18. 

11 U.S. Department of State, About the Bureau of Educational and Cultural Affairs, http://exchanges.state.gov/about.html 
(accessed April 7, 2010). 

12 U.S. Department of State, "Digital Outreach Team," Bureau of International Information Programs, Jan. 2009, 
http://www.state.gov/documents/organization/116709.pdf (accessed April 7, 2010). 

13 U.S. Department of State, "IIP Snapshot," Bureau of International Information Programs, Feb. 2009, 
http://www.state.gov/documents/organization/120493.pdf (accessed April 7, 2010). 
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major cities, staffed by DoS employees, and provided broad access to information. After 
the bombings, many American Centers were closed, moved to embassy compounds, and 
reopened as Information Resource Centers. IRCs still provide information about US 
"foreign and domestic policies; political and legislative developments; defense and 
security matters; and social issues," but they now focus on internet and database 
resources rather than lectures and courses. 14 

Some IRCs provide a high level of on-site access, as in the case of the US Embassy in 
Seoul; however, many restrict physical access and provide limited information. 15 The 
majority of IRCs are located in US embassy compounds and half are limited to 
appointment-only access. In some countries, few IRCs with limited hours may be due to 
the widespread availability of alternative internet resources, but in other regions, such as 
the Middle East, the restriction is most likely due to security concerns. Since moving 
some IRCs to embassy compounds, those in embassies in the Middle East receive six 
times fewer visitors per month than those located off the compound. 16 

DoS also supports American Corners, which are similar to American Centers in that they 
provide local populations with information about American culture and policies; 
however, American Corners have strong differences with American Centers, namely that 
they are much smaller and have substantially fewer programs. Since 2000, the US has 
opened nearly 400 American Corners in approximately 60 countries. These facilities 
operate as partnerships: the US provides equipment and materials while host institutions 
provide the free use of space and the staff to run the American Corners. 17 The staff are 
not managed or paid by DoS. Due to their dependence on foreign nationals and local 
infrastructure, the availability and quality of services provided at the American Corners 
varies widely. 

Various public diplomacy experts have discussed their concern with American Centers 
being downsized and moved into embassy compounds. It appears DoS is trying to reverse 
the downsizing trend. The Fiscal Year 201 1 Department of State Executive Budget 
Summary notes that DoS will open American Centers "outside of US embassies and 



14 Embassy of the United States, London - UK, Public Affairs Section, http://www.usembassy.org.uk/ukpa_irc.html (accessed 
June 27, 2010). 

15 Embassy of the United States, Seoul - Korea, Information Resource Center, http://seoul.usembassy.gov/irc.html (accessed 
June 27, 2010). 

16 Senate Committee on Foreign Relations, "U.S. Public Diplomacy-Time to Get Back in the Game," Federation of American 
Scientists: 2010 Congressional Reports, Feb. 13, 2009, http://www.fas.org/irp/congress/2009_rpt/pubdip.pdf (accessed Sept. 
24, 2009), p. 2. 

17 Embassy of the United States, Budapest, Hungary, Cultural Affairs Office, 
http://hungary.usembassy.gov/american_corners2.html (accessed Oct. 15, 2009). 
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consulates to revitalize the official American presence in foreign countries and to provide 
high-tech venues for people-to-people engagement." 18 



Information Resource Centers - Locations and Access 



Region 


IRC 


IRCs Located 


IRCs with 


IRCs with No Access to the 




Total 


on Embassy 
Compounds 


Public 
Access by 
Appointment 
Only 


Public 


Africa 


37 


21 (57%) 


9 (24%) 


0 


East Asia 


28 


18 (64%) 


15 (54%) 


3 (Sydney, Singapore, Hong 
Kong) 


Europe 


55 


43 (78%) 


30 (55%) 


1 1 (Brussels, Baku, Berlin, 
Copenhagen, Nicosia, Paris, 
Tallinn, The Hague, Moscow, 
Yekaterinburg, Stockholm) 


Middle East 


16 


12 (75%) 


6 (50%) 


2 (Sana'a, Yemen; Beirut, 
Lebanon) 


South/Central 
Asia 


16 


8 (50%) 


8 (50%) 


2 (Karachi and Lahore, 
Pakistan) 


Latin America 


25 


20 (80%) 


19 (76%) 


1 (Bogota, Colombia) 


Total 


177 


122 (69%) 


87 (49%) 


19(11%) 



Source: Senate Committee on Foreign Relations, "U.S. Public Diplomacy— Time to Get Back in the Game," Federation of 
American Scientists: 2010 Congressional Reports, Feb. 13, 2009, http://www.fas.org/irp/congress/2009_rpt/pubdip.pdf 
(accessed Sept. 24, 2009), p. 6. 



18 U.S. Department of State, "Executive Budget Summary," U.S. Department of State, 2010, 
http://www.state.gov/documents/organization/135888.pdf (accessed July 5, 2010), p. 17. 
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Average Monthly Visitors to U.S. Information Resource Centers 




Africa East Asia Europe Middle East South Latin Worldwide 

Central Asia America Average 



■ Visitor/IRC -ON Embassy Compounds ■ Visitor/IRC -OFF Embassy Compounds 



Source/ Senate Committee on Foreign Relations, "U.S. Public Diplomacy— Time to Get Back in the Game," 
Federation of American Scientists: 2010 Congressional Reports, Feb. 13, 2009, 
http://www.fas.org/irp/congress/2009_rpt/pubdip.pdf (accessed Sept. 24, 2009), p. 8. 



Bureau of Public Affairs 

The Bureau of Public Affairs focuses on informing American about the US's foreign 
policy and efforts around the world. 

Office of Policy, Planning and Resources 

The Office of Policy, Planning and Resources, which includes the Evaluation and 
Measurement Unit (EMU), works to evaluate DoS's public diplomacy efforts and advises 
DoS on how best to allocate its resources to meet its objectives. 

Office of Private Sector Outreach 

The Office of Private Sector Outreach works with US private enterprises, such as 
companies, universities, and foundations that have an international presence. DoS 
partners with these organizations to "promote mutual understanding, counter violent 
extremism, empower women business leaders, provide much needed humanitarian relief, 
strengthen international education, encourage health advocacy, and promote social and 
economic development throughout the world." 19 



19 U.S. Department of State, Office of Private Sector Outreach, http://www.state.gOv/r/partnerships/ (accessed Oct. 14, 2009). 
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Survey of Public Diplomacy Professionals and Academia 

To assess the strategic missions of US public diplomacy programs, the research team 
distributed an anonymous survey to its contacts at various embassies. Contacts were not 
limited to Public Affairs Officers, but included any embassy contact familiar with public 
diplomacy programs and DoS's goals for these programs. The team designed the survey 
(see Appendix A) to capture user perceptions and expectations concerning public 
diplomacy themes and issues. 

Survey Questions 

The survey team designed a variety of questions to capture the goals of public diplomacy 
efforts. The team anticipated a high degree of response bias from current DoS employees 
who may want to support DoS. To minimize this bias, the team used combinations of 
direct and indirect questions. The survey asked respondents to state, in a qualitative 
manner, the purpose of US public diplomacy. In an effort to validate the first response, 
the survey included two questions that asked respondents to identify four long-term and 
four short-term goals of US public diplomacy efforts. 

A ranking question (1 indicated "most important" and 5 indicated "least important") 
asked respondents to express opinions regarding goals of public diplomacy. The 
respondents were then asked to identify five ways in which they currently measure their 
own performance in relation to these goals. The survey also included a qualitative 
question in which respondents could state their suggestions to improve the measurement 
of US public diplomacy evaluation. 

Response Rates on Survey 

The purpose of sampling was to construct an accurate composite picture of the target 
population of PD professionals and academics. To get a more representative sample of 
the academia, the project team contacted professors who are members of the Association 
of Professional Schools of International Affairs. In total, the sample included 1 1 
Diplomats-in-Residence at various institutions, 32 Foreign Service Officers, 4 current 
US AID professionals, 14 former ambassadors, and 26 academics. 

Given that DoS officials are the target end users of the evaluation system, they were 
considered the most important target group. The response rate for the DoS professionals 
was 55 percent. The response rate from the academia was considerably lower due to 
logistical difficulties. 

Survey Limitations 

The researchers were aware of the limitations in using a survey as a data collection 
technique for gaining feedback. Surveying current DoS employees presented a potential 
for a large response bias given the sensitivity of the topic for a large segment of the target 



19 



survey participants. An example of a possible response bias is the social desirability bias, 
which affects how employees answer questions regarding their employer; in this case, a 
DoS employee may alter his or her answers to improve DoS's appearance. Researchers 
can limit survey bias by using interviews instead of surveys, which allows the researcher 
to gather information through body language and conversational signaling and change his 
or her style of questioning to elicit the most accurate response from the interviewee. 
Unfortunately, given time and resource constraints, the project team was unable to 
conduct a satisfactory number of semi-structured interviews. However, the researchers 
took a number of other measures to reduce these biases, including but not limited to 
ensuring confidentiality, introducing validation questions, and sending the survey out as 
an independent research body. The researchers felt that the incidence of the problem was 
minimized by these precautionary measures. 

Data Collection and Data Analysis Activities 

The team pre-tested the survey instrument with members of the project team prior to its 
use to ensure the survey program reported the results anonymously and participants could 
navigate the survey easily. The team designed and implemented the survey through a 
web-based platform called Survey Methods. Survey participants were given three weeks 
to complete the survey, at which point the results fed into Microsoft Excel for subsequent 
manipulation. 

In assessing the responses, the survey team first grouped the responses based on shared 
concepts or targeted outcomes. Many of the responses overlapped across column- 
designated topical categories. For example, a response stating that the goal of public 
diplomacy is to disseminate information to foreign publics so they can understand US 
foreign policy better would cover three categories — target population (foreign public), 
input (disseminate information), and outcome (increased understanding of US foreign 
policy). The researchers then used a table frequency analysis to identify the most 
prominent themes used in the responses. Examining the frequency of concepts in the 
survey results served as an important exercise in determining the final outcome 
statements and target audiences. The responses also helped the team identify the short- 
term and long-term goals of the public diplomacy efforts. 

In their responses regarding the qualitative purpose of public diplomacy, 19 percent of 
the survey respondents thought that public diplomacy efforts should reach out to foreign 
governments, 38 percent maintained that they should focus on the general public, and 29 
percent identified elites as the desired target group. Information dissemination, increasing 
understanding, and gaining influence were themes that cut across all questions and 
dimensions. 



Focus Group 

On October 27, 2009, the project team conducted a focus group with the following 
participants: 
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1 . Mr. Carl Chan, Director, Executive Director at United States Advisory Commission 
on Public Diplomacy 

2. Mr. Gerald McLoughlin, Deputy Director, US Advisory Commission PD 

3. Mr. Bill Stewart, Diplomat-In-Residence, LB J School at the University of Texas at 
Austin 

The participants were asked to talk about the following themes: 

• What is public diplomacy? 

• What are some of the measurement issues under the current regime? 

• What would be an ideal measurement system? 

• The information below is a brief overview of the topics discussed in the focus group. 
Discussion 

With the integration of USIA into public diplomacy and the introduction of the 
Government Performance and Results Act (GPRA), Congress has begun focusing on 
results-based strategic communication. The main role of PD is to decrease anti- 
Americanism and increase understanding of US policies. DoS is examining various 
methods for measuring the effectiveness of public diplomacy efforts. Currently, it 
measures nearly all PD effects in the long-term. For example, DoS evaluates Chinese 
relations over a 10 - 15 year range; it also evaluates Fulbright exchanges and participants' 
careers over a long time period. 

One question underlies the evaluation of public diplomacy efforts: is public diplomacy 
worth doing? A second issue is the cost-effectiveness of evaluating PD programs. While 
it is necessary for the US to employ PD efforts to inform and influence other countries, 
measuring the effectiveness of PD, assuming it is possible to do so, may not be cost- 
effective. 



Expert Speakers 

Expert speakers were an important source of information as the research team began 
developing of the measurement framework. The following are some key themes 
discussed in this process. 

The Role of the US Advisory Commission on Public Diplomacy and the Expectations 
from the Project 

David Firestein, Senior Advisor of the US Advisory Commission on Public Diplomacy at 
the start of the project presented the Commission's stance on the impetus for the project 
and gave a first person account of the challenges Foreign Service Officers face regarding 
their PD missions. He also explained how the 1948 Smith-Mundt Act established the 
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Commission as a standalone entity to oversee DoS's public diplomacy efforts. He stressed 
that the goal of PD is always to advance US interests, but current efforts to measure PD 
programs are either inadequate or non-existent. Historically, PD efforts were generally 
not measured, but if they were measured, the review was anecdo tally based. In 1993, 
under GPRA, Congress made the first request for quantifiable results, which has proved 
challenging. 

In addition to having difficulty quantifying PD results, analysts have difficulty discerning 
whether an observed effect is the exclusive result of a specific PD effort or the cause of 
some other event. The "bleed over" effect makes it difficult to determine whether 
observed effects are the result of a PD program or the result of another agency's actions. 

Public Diplomacy Operations, the Washington Perspective 

Former Under Secretary of State for Public Diplomacy and Public Affairs (2005-2007), 
Karen Hughes discussed the changes and challenges for public diplomacy that occurred 
during her tenure. She implemented several mechanisms to help embassies respond faster 
to developments in country and to keep Washington up to date on developments abroad. 
In order to increase dissemination of information in their countries, embassies were given 
a series of talking points they could turn to without having to wait on approval from 
Washington. Additionally, embassies developed weekly briefing papers for use in 
Washington to discuss the major issues and updates from the embassies. This also 
allowed for ambassadors to address issues in other countries. Ms. Hughes also discussed 
efforts to isolate extremists abroad and pointed out some successful PD programs, 
including sending the USS Comfort to cities in Latin America and setting up citizen 
exchanges. She identified polling as one of the tools that was often used to measure PD 
efforts. 

Public Diplomacy Operations, the Embassy Perspective 

Ambassador Engle is the Associate Director of the Robert S. Strauss Center for 
International Security and Law and a member of the faculty at the University's Lyndon 
B. Johnson School of Public Affairs. He talked about several PD efforts, including 
libraries, English language programs, special events such as election night parties, and 
cultural and academic exchanges through physical and virtual means. He focused on 
media outreach and the ability of the embassy to relate to the indigenous culture where it 
is operating. Relevance to the country's culture is one of the important factors to the 
success of PD efforts. PAOs must also carefully choose the method they use to target an 
audience, for instance, television, radio, internet or print media. One key issue 
Ambassador Engle raised is his view that to know us is not necessarily to love us, a factor 
the team needed consider when determining the impact of PD efforts. Understanding is a 
two-way activity, and ultimately DoS should strive to increase favorability toward US 
Policy. 

Current Evaluation Efforts 

Cherreka Montgomery is the Director of the Evaluation and Measurement Unit (EMU) 
within the Office of Policy, Planning and Resources for Public Diplomacy and Public 
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Affairs. She presented her work on the Public Diplomacy Impact project, as well as 
discussed historical efforts to evaluate public diplomacy efforts. She also discussed the 
Public Diplomacy Logic Model. (See Appendix A.) 

Designing Valid and Reliable Survey Instruments 

Mr. Jeff Smith, Opinion Analyst and Polling Expert, discussed key points to focus on 
when developing a survey. Some specific points for designing the survey were to ensure 
the questions are clear and unambiguous. In order to improve the response rate, a survey 
team must get people invested from the beginning of the survey so that they want to give 
their opinion and finish the survey. He also suggested that the number of questions is not 
important, but the length of time it will take respondents to complete the survey is 
extremely important. 



Overview of Results 

From its research of current and recent DoS public diplomacy documents, interviews 
with public diplomacy professionals and focus groups, and survey results from 
professionals in the PD field, the research team found these definition, mission, and 
understanding of public diplomacy differs according to each person's perspective. As an 
external group without full access to PD programming information, the research team 
found it difficult though not impossible to determine consistent, clear, and quantitative 
means to measure the effects of PD. 

Key Themes 

Information Dissemination 

Sixty-two percent of the respondents mentioned disseminating information on US foreign 
policy and goals as one of the purposes of public diplomacy; however, only 10 percent 
regarded this as a long-term goal. Forty-eight percent mentioned the importance of 
disseminating information about US culture as a goal of public diplomacy with 8 percent 
highlighting this as a long-term goal. 

Increasing Understanding 

Twenty-four percent of respondents mentioned increasing understanding regarding US 
foreign policy and goals as one of the purposes of public diplomacy. Fourteen percent 
regarded this as a long-term goal. Nineteen percent mentioned the importance of 
increasing understanding about US culture as a purpose of public diplomacy; 1 1 percent 
highlighted increasing understanding as a long-term goal. 

Increased Influence 

Forty-three percent of the respondents identified influencing foreign audiences to comply 
with US foreign policy and goals as one of the purposes of public diplomacy, while only 
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24 percent regarded this as a long-term goal. Ten percent mentioned the importance of 
influencing foreign audiences to be sympathetic to US culture as a purpose of public 
diplomacy; 7 percent highlighted this as a long-term goal. 

Understanding US Policy 

The ranking question validated the importance of understanding US policy as the most 
important long-term goal with 50 percent of respondents citing it as the most important 
long-term goal. The other goals were ranked in the following order: increasing favorable 
views of the US, increasing understanding of US culture, increasing the US's security, 
and strengthening US ties with foreign governments. 



Current Performance Measures of PD 

The OMB currently has 21 PD performance measures, the majority of which are outputs, 
not outcomes. In 2007, the DoS issued the Public Diplomacy Logic Model, which 
identifies programs, outputs, and outcomes. It describes the long-term outcomes reflected 
in the behavior of the target audience as demonstrated by the following conditions: 

1 . Increase in democratic values and support for political rights and civil liberties in 
key foreign countries and publics. This information is collected from Freedom 
House. 

2. Create positive change in foreign publics or audiences' appreciation and 
understanding of US society, values and policies. 

3. Foreign media outlets have greater clarity and understanding of the US's role in 
global affairs and the US's foreign policy goals demonstrated by editorial and 
opinion commentary. 

(See Appendix A.) 

Public Diplomacy Impact Project 

In 2009, the Office of Analysis, Evaluation and Performance Measurement in the Office 
of Policy, Planning and Resources performed a PD Impact assessment for public 
diplomacy. It surveyed two groups of locals (PD program participants and non- 
participants) in eight countries: Ecuador, Germany, India, Indonesia, Morocco, Occupied 
Palestinian Territories, Turkey, and South Africa. The survey attempted to identify the 
effect of PD programs on participants by comparing their responses to non-participants in 
four key areas: 

1 . Understanding, favorability, and the role and influence of the United States in the 
world 

2. Satisfaction with PD programming 

3. Participants' and non-participants' ideas for strengthening the US-target country 
relationship 
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4. Opportunities and receptivity to future PD engagement 20 

One example area of the survey's questions involved questions that asked responders to 
rank his or her understanding of the US on various cultural, governmental, and political 
questions. The other areas followed the similar approach of asking ranking questions and 
comparing participants' responses to non-participants' responses. The assessment also 
compared participants' responses across all the countries analyzed. 

The research team's concerns with the PDI method are that it is susceptible to bias 
(sample bias and selection bias), which the assessment acknowledges as a possibility; it 
gives a snapshot or a perspective from one point in time — it tells you participants' 
perception at one particular moment; and as an aggregate tool, it cannot be used to 
identify specific effective programs or efforts. 

The PDI team surveyed participants and non-participants at one point in time. Not 
comparing participants' perception prior to the PD event and after the PD event leaves the 
possibility that some other event influenced their opinion. The evaluators used a control 
group, non-participants, to attempt to cure this issue, but the possible selection bias raises 
another problem. People who participate in a PD program may already have a favorable 
opinion or more understanding of the US in comparison to non-participants. Also, asking 
participants after they have participated in a program whether they think they have an 
increased understanding of the US or increased favorability towards the US is a 
qualitative assessment, rather than a quantitative assessment. 

Mission Activity Tracker (MAT) 

The MAT was launched in 2007 to provide a method for tracking program costs and 
activities and evaluating whether DoS and embassies are allocating resources according 
to their strategic goals. It tracks activities and the number of projects or information types 
that IIP and overseas missions complete in each fiscal year. The MAT reports on 
activities, but does not draw a connection between the activities and the Office of PD and 
PA's strategic mission. It gives no relative information (such as an increase or decrease in 
outputs and outcomes) that administrators can use to determine whether a program is 
achieving its objectives. It is a good interface for collecting information, but it needs a 
supporting tool that allows administrators to evaluate the information collected. 



20 ICF Macro, Public Diplomacy Impact [PDI) Project: Country Synopsis: South Africa, (ICF Macro, 2009), p. 1. 
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Chapter 3. PD-MAP 



The PD-MAP is designed to complement ongoing evaluation efforts such as the PDI 
project and MAT. It offers a tool for PD officers to use to evaluate a target audience's 
perception, favorability, or understanding prior to and after a PD effort in order to 
identify the PD program's direct impact. Each embassy will have a different goal for its 
work, so the PD-MAP offers a flexible framework that can be modified to fit an 
embassy's strategic mission. 



The Model 

In developing the PD-MAP, the project team decided to use a multi-criteria decision 
making model, MCDM. MCDMs allow the user to compare factors that go into deciding 
between various possible actions. For example, when a PD officer is deciding whether to 
allocate more resources towards embassy cultural events or exchange programs in order 
to increase the general population's deep understanding of the US's culture, he or she 
places a high priority on the general population as a target audience, deep 
comprehension, and US culture. These three factors — population, level of understanding, 
and topic — are the factors that a PD officer might value highly in this strategic goal. 

A PD officer is able to adjust the model based on his or her personal risk assessment and 
priorities for different factors. This model allows a user to analyze multiple variables to 
determine a relative score of a particular alternative. In calculating the scores, the model 
uses the best and worst potential performance values and the level of acceptable risk for 
each element. The model incorporates weights for all program factors to determine the 
program's overall effectiveness in reaching its strategic goal. 

For example, when determining whether a PD program leads to increased understanding, 
the user can weight the factors differently based on the specific goal of the program. In 
the case of increasing the understanding of US culture among the general population in 
Egypt, a PD program may have the goal of providing accurate, accessible information to 
the general population. High comprehension is a long-term goal, but information 
dissemination and reception are the short-term goals. In that case, the user can weight 
information and dissemination more heavily than comprehension and revise as the 
program matures. 

The multi-criteria decision-making model is preferable for a multidimensional program 
such as public diplomacy since flexibility is built into the model. If an analyst cannot 
collect data points, the model does not count the lack of information against the relative 
score. The score can be used to identify changes in the effect of a PD program over time 
in a specific country. The model has no inherent bias and the results are reliable and 
reproducible. Additionally, the model accounts for various units and scales. Another 
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benefit is that the model utilizes readily available spreadsheet software and is relatively 
time-efficient to use. 

The project team used three desired outcomes of public diplomacy, described below, and 
weighted each equally in determining performance. DoS or EMU can alter the weights 
based on an embassy's strategic mission. Each of these outcomes has multiple sub- 
outcomes that contribute to the outcome's score. The sub-outcomes are currently 
weighted according to the survey results the research team received in the fall, but can be 
adjusted according to a program's goals. In turn, each of the sub-outcomes has multiple 
performance metrics or measurements, as shown in the next chapter. Each measurement 
is also weighted individually, which, again, can be adjusted. Based on an individual 
embassy's strategic mission and type of programs, the measurements' weight will vary. 
This makes the model very tunable and sensitive to differences between strategic 
missions, countries, and PD programs. 

To determine the score of a performance metric, the actual performance value is 
subtracted from the best possible performance value and then divided by the range of 
potential performance. This number is multiplied by the relative weight. The pertinent 
performance metric scores are then combined to create the sub-outcome score. This score 
is weighted and combined with the other sub-outcomes to contribute to the overall 
outcome score. 



Desired Outcomes for PD 

Based on its survey of PD officers and experts, interviews with PD experts, reviews of 
PD analysis, and reviews of current PD practices and programs, the research team 
identified the following three outcomes as the primary goals for PD efforts: 

Increasing Understanding of the US 

Understanding the US means that a foreign audience comprehends both the US's 
government policies and its culture, as well as how the two influence each other. The 
project team determined that understanding is a three-step process: dissemination of 
information, reception of the information, and comprehension of the information. 

Increasing Favorability toward the US 

Favorability is the approval of US policy or culture — the opposite of Anti- Americanism. 
It exists across multiple spectra and is demonstrated by the actions, behavior, and 
opinions of foreign audiences. 

Increasing US Influence in the World 

The US's influence refers to the capacity or power of the United States to directly or 
indirectly change the actions, behavior, and opinions of foreign audiences. 
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Target Audiences 

Diplomacy refers to government-to-government relations, while public diplomacy refers 
to a government's relations with foreign citizens. The PD audience is not a homogenous 
group and PD programs are designed to target diverse groups within a larger foreign 
society. Since PD is generally intended to inform and influence a society so that citizens 
will in turn influence their government to support US policies and interests, the vast 
majority of PD programs target citizens. Citizens can be broken into two categories — 
general population and elites. The make-up or demographics of these two groups will 
vary from country to country, but they are generally differentiated by their ability to 
influence the government. The general population is influential en masse, while elites 
have influence through various higher-level channels. DoS designs its programs to 
interact with both groups based on their interests, education levels, and areas of influence. 

General Population 

Beginning with the common understanding that PD efforts target the general population, 
the project team discovered that measuring PD's impact on the general public alone did 
not capture the full realm of PD efforts or intended targets. Therefore, it separated the 
general population and defined them as the elites and middle-lower socioeconomic 
groups who have limited access to political influence. The demographic of this group will 
depend upon the country. 

Elites 

The team defined elites as religious, social, cultural, business, and political leaders, as 
well as members of the media who have influence over the public or government's 
opinion towards the US. 

Government Officials 

DoS also has a few PD programs that target government members, who are influential 
members in every society. As stated previously, interaction with government members is 
generally considered diplomacy, but DoS also runs programs that target emerging 
governmental and political leaders in the hopes of building an understanding and 
relationship with them as they rise to positions of influence. These include but are not 
limited to events and exchange programs for government officials. 

Some programs target all or combinations of the three groups. The model allows users to 
review programs that target multiple audiences and evaluate their effect on each audience 
type. The next chapter will explain how the research team took this information — desired 
outcomes and target populations — and created an evaluation model that will identify PD 
efforts' impact on both. 
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Targeted Areas 



Based on the project team's background research and evaluation of the goals of current 
PD programs, the project team focused on evaluating the ability of PD efforts and 
programs to increase understanding, favorability, and/or influence in five areas: 

■ US culture 

■ US foreign policy 

■ US security 

■ Foreign economic policy and economic ties with the US 

■ Foreign climate change policy. 

DoS has different goals or desired outcomes in relation to these areas, which will be 
delineated in the following chapters. 
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Chapter 4. Understanding 



Components of Understanding 

The project team identified "increasing understanding" as a desired outcome of PD 
efforts. Understanding is a broad concept that can be broken into three distinct and 
progressive steps, which should be measured individually and then combined to 
determine whether the PD effort is providing the level of communication necessary to 
increase comprehension. The steps are: 

1) Dissemination of the information 

2) Reception of the information 

3) Comprehension of the information 

Measuring step one informs PD officers how many informational products of various 
types the PD effort produces. Measuring step two tells the PD officer whether the 
communication approach (dissemination method or type of informational product) is 
appropriate for the target audience. Measuring step three tells the PD officer whether the 
informational product is conveying the PD effort's message in an accessible way. 

Step three's measurement only tells the PD officer the level of comprehension of the 
people who received the information. If the reception or dissemination numbers are low, 
but the message was clearly comprehended by the recipients, measuring only step three 
could mislead the PD officer into thinking that the communication effort was effective at 
increasing the target audience's understanding. If the PD officer's goal is to increase 
understanding among a large population, increasing only 30 people's comprehension of 
the US would fail to meet that goal. Therefore, PD officers should measure each step 
independently to identify any problems in the way a PD effort is delivered. To assess the 
overall effort, the PD officer should weight the results for each step according to the 
strategic goal, and then combine the scores to get an accurate aggregate score. 

The following section provides a more detailed explanation of each step, its contribution 
to the general concept of understanding, and an example of how it can be measured. 
Detailed examples of measurements can be found in Appendices C. 
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Understanding 




Example metrics available in Appendix X 



Dissemination 

The first step in conveying information is dissemination. The appropriate method of 
dissemination will depend on the PD effort's goal, the type of information, and the 
characteristics of the target audience. Current PD methods include but are not limited to 
informational pamphlets, cultural events, class lessons, informational forums, educational 
material at American Corners or IRCs, embassy staff speaker events, and cultural, 
professional, and educational exchanges. 

In this step, a PD officer will determine the best dissemination method — the method that 
gives the target audience access to the information. The PD officer will define the goal of 
the PD effort, indentify the target audience, and determine the best way to deliver the 
information. For example, if the goal of a PD effort is to educate journalists on how to 
use the internet for research, then a PD officer might choose an educational forum held at 
an IRC as the best method for dissemination. A PD effort with the goal of educating a 
large population on America's musical culture might use concerts for the general 
population as a more effective method than handing out pamphlets at American Corners. 
The model can be used to evaluate how well an embassy performed in making the 
information available. 

Example measurement of dissemination: 

• Measurement : Relative change in the number of policy related informational forums 
directed toward members of foreign governments in a FY over the number of similar 
events held in the previous FY. 
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Reception 



Reception is the measurement of the number of people who receive information from a 
PD effort. It follows dissemination in that the quantity of people receiving information 
depends on appropriate delivery of that information. For example, if an embassy holds a 
cultural event targeted at the general population or the elite population, but attendance is 
less than desired for the goal of the PD effort, the event may not have been promoted 
widely enough or was not designed appropriately to attract the target audience's interest. 
This measurement will help determine whether the dissemination efforts, though 
sufficient in scale, were appropriate for targeting the intended audience. 

Additionally, if the dissemination and reception scores are high for a given event, but the 
comprehension score is low, then the type of event or informational approach may not be 
appropriate for the target audience. 

Example measurement of reception: 

• Measurement : Relative change in the number of government officials who attend a 
policy related informational forum from the number of government officials who 
attended a similar policy forum the previous FY. 

Or 

• Measurement : Relative change in the number of government officials who attend a 
policy related informational forum from the desired number or invited number of 
governmental officials. 

Comprehension 

Comprehension is the amount of information presented through a PD effort that is 
retained by the audience. Analysts must establish a baseline measurement of 
understanding prior to a PD effort to get an accurate measurement of its effectiveness in 
facilitating the audiences' retention of the presented information. Increased 
comprehension might include the recipient of a PD effort correctly identifying more 
elements of the disseminated information than he or she identified prior to receiving the 
information. For example, if the PD effort is a lesson on US culture presented through 
publications at an American Corner, the audience should be assessed prior to reading the 
materials and after reading the materials. For a Fulbright program, program managers 
should assess recipients' understanding of the US prior to entering and after leaving the 
program. 

Example measurement of comprehension: 
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• Measurement : Relative change in the number of tenets official members of a foreign 
government identify after a policy related informational forum from the number 
identified prior to the event. 



Target Audience 
General Population 

The methods to measure increased understanding for the general population will depend 
on the general population's education and modes of communication accessible to the 
general population. The PD officer overseeing a program will determine the best method 
for assessing the general population's understanding prior to and after an event or PD 
effort. In countries where the general population has high literacy rates, the PD officer 
may distribute questionnaires. In countries with low literacy rates, the PD officer may 
conduct interviews. 

Example measurements of PD efforts targeting the general population regarding US 
culture: 

• Measurement: Relative change in the number of US cultural events targeted towards 
the public in a FY over the number in the previous FY. 

• Measurement: Relative change in the number of general public attendees to US 
cultural events over the desired or estimated number of people. 

• Measurement : Relative change in the number of surveys that correctly identify more 
than half of the facts regarding US culture than the surveys collected prior to the 
cultural event. 

Elites 

PD efforts targeted at elites take into consideration their education levels, their areas of 
interest and influence, and the embassy or DoS's mission. Each aspect will, of course, be 
dependent on the country and the demographics of its elite population. For example, 
programs that target journalists will vary from country to country based on journalists' 
education and area of interests. 

To measure whether comprehension increases among the elite after PD efforts, PD 
officers can assess comprehension through surveys, reading news articles or editorials 
written by recipients, attending community speeches by recipients, or conducting 
interviews, among other options, before and after a PD effort or event to determine 
whether the effort or event was the cause of the change. 

Example measurements of PD efforts targeting elites regarding US policy are: 
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• Measurement: Relative change in the number of policy related informational sessions 
targeted towards journalists in a FY over the number in the previous FY. 

• Measurement: Relative change in the number of journalists attended a policy related 
informational sessions and number who were invited to the event. 

• Measurement: Relative change in the number of editorials correctly delineating US 
policy in a quarter over the number in the previous quarter. 

Government Officials 

Members of government are often included in exchange programs, informational events, 
and embassy cultural events. Measuring the number of exchange opportunities or events, 
the number of participants, and the level of understanding of US policy and culture that 
results from this participation can inform the DoS and PD officers on whether they have 
effectively designed, promoted, and implemented these events. 

Example measurements of PD efforts targeting members of the government regarding US 
policy: 

• Measurement : Relative change in the number of exchange opportunities for 
government members of the government in a FY from the previous FY. 

• Measurement : Relative change in the number of government participants in the 
exchange programs in a FY from the previous FY. 

• Measurement : Relative change in the number of tenets of US culture or policies that 
participants correctly identify prior to and after an exchange program. 

Targeted Policy Areas 

Based on the background research of PD efforts and the focus of current measurement 
efforts within the DoS, the project team focused the measurement effort on two policy 
areas: culture and US foreign policy. Public diplomacy programs try to increase 
understanding in these two main areas. PD activities can be divided into activities 
focused on US culture and those focused on US foreign policy. 

Culture 

One of the primary goals of public diplomacy is to increase understanding of US culture 
abroad. The basis of the goal is "to promote the United States, its culture, [and] society 
... to encourage greater mutual understanding, . . . [and] to foster a sense of common 
interests and values between Americans and people of different countries, cultures and 
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faiths." 21 PD efforts that increase understanding of US culture are particularly important 
in light of the proliferation of American pop culture available abroad, and the 
misconceptions that can arise from exposure only to pop culture. Also, given the US's 
significant security concerns and economic interests abroad, DoS uses public diplomacy 
to increase foreign populations' understanding of the US in order to halt conflict and 
encourage cooperation. 

Example measurement targeting understanding of US culture: 

• Measurement : Relative change in number of American cultural events (music shows, 
author book reading events, art shows, athlete speaker events, etc.) hosted by the 
embassy in a FY from the previous FY. 



US Foreign Policy 

Another key aspect of public diplomacy is increasing foreign populations' understanding 
of US foreign policy decisions. DoS's PD framework lists the objective of promoting US 

22 *~ 

"policies to encourage greater mutual understanding." This aspect is important to 
combat misinformation regarding US foreign policy especially in countries with 
governments that deliberately distort the reasons behind US policy decisions for political 
reasons. PD efforts focused on understanding of US foreign policy decisions are the US's 
opportunity to promote its reasons for policy decisions. 

Example measurement targeting understanding of foreign policy: 

• Measurement: Relative change in the number of policy related informational forums 
directed toward members of foreign governments in a FY from the previous FY. 



21 United States Department of State, "The Budget in Brief: Fiscal Year 2010," United States Department of State, 2009, p. 23. 
http://www.state.gov/documents/organization/122511.pdf (accessed April 12, 2010). 

22 Ibid., p. 23. 
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Chapter 5. Favorability 



Favorability, the second outcome, attempts to measure how well a foreign public regards 
the US. The simple question that the outcome should attempt to answer is: "Does the 
target audience like the US more now than before the implementation of the PD events or 
programs." The below diagram illustrates how Favorability could be measured 
subsequent to PD efforts. 



Favorability 




Example metrics available in Appendix F 
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Dimensions of Favorability 



The diagram below shows five different definitions of favorability. The degree to which 
an individual feels favorable about a subject may never be consistent. In order to account 
for varying degrees of favorability, it is important to know possible definitions people 
might associate with this term. 



Levels of Favorability 



Favorability 



Approving 



Expressing approval 
or administration 



Promising 




Winning approval or 
favor 



Suggesting future 
improvement or 
good results 



Acting In i 
beneficial way 
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Target Audience 
General Population 

Measuring favorability among the general population will vary greatly from country to 
country as publics express their positive or negative opinions differently. In France, 
citizens are accustomed to protesting in large numbers while in Kenya the police are 
quick to disperse large crowds. The measurements for each type of society will differ 
according to the level of the general population's freedom of expression. 

Example measurement of the effect of a PD program or effort on the general population's 
opinion of the US: 

• Measurement : The change a PD program participants' favorable opinion towards the 
US prior to and after attend an event or participating in a program. 

Elites 

A wide spectrum of individuals are considered elite throughout all countries. Therefore, 
measuring the change in their opinion towards the US as a result PD programs will 
depend greatly on their form of expression and the program or event in which they 
participate. Below are just a couple of measurement examples of ways PD officers can 
measure the chance in elites' opinions towards the US. 

• Measurement : Relative change in the number of people referred to an informational 
forum, speaker event, or exchange program by a colleague or friend in a FY over the 
previous FY. 

• Measurement : Relative change in the number of positive articles written about the US 
by journalist who have attended the Edward R. Murrow Program for Journalists in a 
FY over the previous FY. 

• Measurement : Relative change in the number of positive editorials written by 
community leaders who attended an informational forum or heard a speaker from an 
embassy in a FY over the previous FY. 

Government 

The importance of changes in favorability among government officials will vary across 
countries depending on the strength of governmental institutions in each individual 
country. Regardless of relative institutional strength, government officials are uniquely 
positioned to influence their government's decisions, whether directly or indirectly. PD 
officers can assess the favorability of government officials towards the US before and 
after they participate in a PD program; however, the possibility of bias — government 
officials saying or agreeing with positive statements about the US to curry favor — is high 
for political reasons. Pre- and post-surveys may be one option, but they should be 
supported by a second measurement. Due to their political positions, it may be difficult 
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for government officials to express their opinion of the US. If available, however, this 
would be a strong measurement of the effectiveness of a PD program. 

• Measurement : Relative change in the number or level of positive statements 

government officials make prior to and after participating in a PD event in a FY over 
the previous FY. 



Target Policy Areas 
Security 

The team's survey results show many DoS employees consider increasing Americans' 
security in the US and abroad a top priority goal for PD programs. As a major power in 
the world, the US's domestic and foreign policies affect many countries and often trigger 
unpopular views abroad, which can lead to violence against Americans. PD programs 
offer an opportunity for the US government to reduce anti- Americanism by engaging 
foreign populations and showing that the US is concerned about the effects of its actions 
and works to positively impact foreign population's lives. 

Evaluating whether a PD program is successful in increasing US security will depend on 
the target population and their ability to threaten Americans' security. Many factors are 
involved in protecting Americans' security, and any one program will most likely be 
insufficient to fulfill the task. Measuring the level of pro- or anti- American sentiment 
prior to implementing a set of PD programs can help PD officers determine what methods 
are most effective. 

Example measurements of PD programs effectiveness at reducing threats to US security 
are: 

• Measurement : Relative change in number of protests against the US in a FY over the 
number of protests in the previous FY. 

• Measurement : Relative change in number of editorials that express strong anti- 
American sentiment in a FY over the number of editorials in a previous FY. 

• Measurement : Relative change in number of public statements by government 
officials against the US in a FY over the number of protests in the previous FY. 



Economy 

In a highly globalized world, the US economic cycle affects many countries. DoS 
employs PD programs such as professional exchanges, business related info sessions, or 
job development conferences to improve foreign countries' economic capabilities and 
strengthen the US's international trade ties. PD informational forums can also inform 
foreign publics about the nature of and reason for US economic policies. Most PD efforts 
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working to engender a favorable opinion of the US's economic policy are targeted 
towards elites and the government. However, some programs also attempt to explain the 
benefits of the US's economic structure so to the general population. Below are examples 
of measurements for programs that target all three groups of a foreign society. 

Example measurements of PD programs effect on increasing target groups' favorable 
opinion of US economic policy are: 

• Measurement : Relative change in change in number of protests against American 
companies or American economic policy in a FY over the number of protests in the 
previous FY. 

• Measurement : Relative change in change in number of editorials that strongly 
criticize American companies or American economic policy in a FY over the number 
of protests in the previous FY. 

• Measurement : Relative change in change in number of US imports into a country in a 
FY over the number in the previous FY. 



US Culture 

The Bureau of Educational and Cultural Affairs (EC A) and the Bureau of International 
Information Programs (IIP) operate many programs that are designed to share America's 
culture with the world. According to the Cultural Programs Division of the ECA, 
"American arts and culture reflect our nation's democratic principles, showcase the 
diversity of our society, and provide an excellent vehicle to engage and connect with 
audiences abroad." 23 DoS hopes to engender positive feelings about the US by sharing 
American culture through music, art, and literary events. 

Example measurements of culturally related PD programs effect on foreign audiences' 
favorable opinion of the US are: 

• Measurement : Relative change in change in number of foreign artists who have 
participated in a cultural exchange program and then collaborate with American 
artists or incorporate American style art and music into their work in a five-year 
period over the previous five-year period. 

• Measurement : Relative change in change in number of foreign audiences who attend 
US embassy sponsored film festivals in a FY over the previous FY. 



23 U.S. Department of State, Cultural Programs Division, http://exchanges.state.gov/cultural/index.html (accessed July 8, 
2010). 
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Chapter 6. Increasing the US's Influence in the World 



Explanation of Influence 

One of the three primary outcomes used to measure a public diplomacy program's impact 
on US interests is whether a particular PD program can increase the United States' ability 
to influence a population or government's opinion, behavior, or action. For the purposes 
of this model, influence is defined as the capacity or power of the US to compel foreign 
audiences to either actively support US interests or at least not act against them, 
particularly in the case of US security. 

The influence outcome is linked to the favorability outcome in that PD officers work to 
engender positive feelings towards the US, which ideally will lead PD participants to 
either support US interests or move from anti- American rhetoric or behavior to a neutral 
state. Favorability is an outcome or goal in itself, but influence is the next level of 
engagement. 

Diplomacy works to directly influence foreign governments, whereas public diplomacy is 
the effort to indirectly influence foreign governments through their citizens. The vast 
majority of PD efforts are targeted at the general population or elites rather than 
government officials. DoS works to build support for the US from the ground up — DoS 
builds a relationship between the US and civilians so that civilians will pressure their 
government to support US interests. 
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Influence 




Example metrics available in Appendix G 



Target Audience 
General Population 

Since PD efforts have spillover effects on participants' families, friends, and colleagues, 
its ability to influence foreign audiences can be widespread and difficult to measure in 
terms of actions by participants. PD officers should attempt to measure the direct effect 
of PD on participants, but also evaluate the US's indirect influence on the public and the 
government. When measuring a PD effort's effect on the general public's behavior and 
actions, PD officers can look at similar measurements as those described in the 
favorability section. In some cases, actions are the expression of feelings, so reviewing 
the number of protests in a country, for example, can tell a PD officer how the population 
feels and the lengths to which they are willing to go to influence their government. A PD 
officer must also consider the general public's capabilities for influencing its government. 

Example measurements of PD efforts' ability to influence a general population are: 
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Measurement : Relative change in number of anti-US protests against the US in a FY 
over the number of protests in the previous FY. 



• Measurement : Relative change in number of pro-US statements reported in 
newspapers, radio and television shows, and major social websites in a FY over the 
number in the previous FY. 

Elites 

Elites often have greater opportunities to influence the general population or government 
to take action. A journalist can advocate for a government to support the US in a climate 
change initiative. A religious leader or community can motivate the general public to 
boycott American products, protest for greater civil rights from the government, or 
exhibit greater tolerance for other religious beliefs. 

Example measurements of PD efforts' ability to influence elites are: 

• Measurement : Relative number of public speeches by a PD event participant that 
express strong pro-American sentiment in a FY over the number in a previous FY. 

• Measurement : Relative number of public statements by government officials against 
the US in a FY over the number of protests in the previous FY. 

Government Members 

PD programs that target government members attempt to influence decision-makers who 
direct a country's economic, cultural, environmental, or security policies. PD programs, 
such as the International Visitor Leadership Program (IVLP) familiarize foreign 
government members with US policies and culture and encourage them to adopt or 
support US policies. The IVLP introduces political leaders to American political leaders, 
so they can share ideas and learn from US practices. Informational forums or events held 
at embassies perform similar functions. PD officers can assess whether these programs 
effectively influence government members by evaluating changes in a country's domestic 
and foreign policies. 

Example measurements of PD efforts' ability to influence members of government: 

• Measurement : Relative change in number of dissidents jailed for speaking out against 
the government in a FY over the previous FY. 



Targeted Policy Areas 

The metrics for each targeted policy area or sub-outcome were chosen based on how they 
might highlight any improvement of the US's ability to influence a particular audience in 
a given country. The following are the primary areas in which the US works to establish 
influence with foreign audience. 
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Security 



The team defined national security as the highest priority area for the US to influence 
foreign audiences. The US government's foreign policies sometimes create animosity in 
the world, which threatens the safety and security of government and business employees 
working abroad, citizens who are traveling abroad, or American investments abroad. DoS 
employs PD efforts to reduce the level of anti- American sentiment or provide 
opportunities for foreign audiences to find alternative focuses for their energy, such as 
employment and greater interaction with their own governments. DoS is concerned with 
determining whether PD efforts are effective at performing this role. 

Example measurements of US's ability to influence foreign audience to support the US's 
domestic or international security are: 

• Measurement : Relative change in the number of statements supporting violence 
against the US that are reported in newspapers, radio or televisions broadcasts, or 
websites or blogs in a FY over the previous FY. 

• Measurement : Relative change in the number of anti- American groups or 
organizations that are active in a country in a FY over the previous FY. 

• Measurement : Relative change in the number of pro-American statements from 
government members reported in a FY over the previous FY. 

Culture 

Through educational exchanges, informational programs, American Corners, 
presentations on US struggles for civil rights, and programs such as the Democracy 
Commission Small Grants Program, DoS attempts to influence foreign audiences to 
either adopt or influence their government to adopt certain American values such as 
respect for human rights, democracy, freedom of speech, and non-discrimination on the 
basis of gender, race, or religion. The purpose is not to homogenize societies around the 
world, but to share American values that can help societies enjoy social and political 
stability. Additionally, since these values are the underpinnings of American culture, the 
US can build stronger relationships with countries that share these values. Influencing 
governments and societies to adopt values is a longer-term process than increasing 
favorability, so the evaluation process may occur over longer periods than fiscal years. 

Example measurements of the effects of PD on influencing foreign audiences to adopt 
certain American values are: 

• Measurement : Relative change in number of racial or gender-based attacks that occur 
within a geographical region or within a population group where DoS has targeted PD 
programs on diversity and respect for human rights over the course of a five year 
period. 
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• Measurement : Relative change in number of NGOs or civil society organizations that 
promote rights for women or marginalized ethnic groups over a five year period. 

• Measurement : Relative change in number of laws or government programs that 
support equal rights for women and marginalized ethnic groups over a five year 
period. 



Environment 

Through Regional Environment Offices (REO) located in US embassies, Environmental 
Affairs Specialists work to change the actions, behavior, and policies of foreign 
populations and governments regarding environmental issues. Environmentally degrading 
practices in many countries threaten the health and livelihood of its citizens, as well as 
the health and security of the world. DoS uses educational programs to teach foreign 
audiences about the risks of pollution and destroying ecosystems, as well as conservation 
and environmental techniques that benefit everyone. As stated previously, changing a 
population's behavior and actions is a long-term process, so the ability of PD programs to 
influence environmental practices requires long-term measurements. 

Example measurements of the effects of PD on influencing foreign audiences to adopt 
environmentally sustainable practices are: 

• Measurement : Relative change in number of hectares of rainforest or grassland that 
are destroyed for agricultural production over the course of a five year period. 

• Measurement : Relative change in the number newspaper articles and radio/television 
shows that discuss environmental risks and promote sustainable living methods over a 
five year period. 

• Measurement : Relative change in number of laws or government programs that 
protect natural resources over a five year period. 



Economy 

Using public diplomacy, DoS promotes economic growth within countries in order to 
create stability and opportunities for trade with the US. Educational and professional 
exchanges, economic forums, professional development workshops, and even English 
language programs help the US build trade relationships with other countries. As with US 
security, PD programs also help influence other governments to support US international 
economic policies such as multi-lateral trade agreements. Most PD programs focus on the 
elites and foreign governments in this goal, but the US also has programs that work to 
increase economic opportunities for the general population. The purpose of these 
programs is usually to promote stability in countries and US security, however, US 
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companies benefit from skilled labor so PD programs targeted at general populations also 
promote US economic interests. 

Example measurements of the effects of PD on influencing foreign audiences to develop 
practices that align with the US's economic interests are: 

• Measurement : Relative change in number of workers in US-preferred industries fields 
who participated in embassy ESL programs over the course of a five year period. 

• Measurement : Relative change in number of partnerships created between foreign and 
US companies after economic or business related forums over a five year period. 

• Measurement : Relative change in number of trade agreements created between the 
US and a foreign government over a five year period. 
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Chapter 7. Model Implementation 



Application of the Model in the Field 

While PD programs vary significantly based on the target audience and their strategic 
mission, this model — using the three outcomes shared across all programs — creates a 
common scale for measuring the effectiveness of all PD programs in achieving these 
outcomes. The model also allows the evaluator to modify the measurements (based on 
standard format) to fit all bureaus, offices, and departments' PD programs and adjust the 
framework to give an accurate, quantifiable assessment of each individual program. 

The team envisions that the Evaluation and Measurement Unit (EMU) will work with 
embassies and PD departments to develop measurements that take into consideration the 
limitations for data gathering, which vary from country to country. The EMU and PD 
officers in the field already gather data in the MAT and create PDI reports in a few select 
countries. This framework allows the two efforts to be incorporated into an evaluation 
process that can be used in every country for every PD program. PD officers will gather 
the necessary data and place it into an interface like the MAT, which then will feed it into 
PD-MAP. PD-MAP will then calculate the scores for each PD program, based on the 
ascribed weights developed by the EMU and relevant PD bureaus. The scores should 
accompany informed, expert opinion of PD programs in each country. 

Cost effectiveness 

The model allows stakeholders to examine costs per program for each embassy, which is 
an important step in deciding how to allocate resources for PD programs based on the 
Office of Public Diplomacy and Public Affairs' strategic goals. With the help of PD- 
MAP, the EMU can combine cost-effectiveness measurements with program 
effectiveness measures. Some programs take years to show an effect, so DoS should 
consider its long-term and short-term goals when incorporating the cost-benefit analysis. 

The team also considered cost-effectiveness in regards to the time, manpower, and 
financial resources that goes into PD-MAP. Much of the same data that is collected in 
the MAT can be used in PD-MAP. Also, the sample measurements provided show that 
the EMU can create additional measurements that will not require substantial additional 
effort by PD officers. The research team did not have full access to the MAT, but it tried 
to include sample measurements that can be gathered at the same time as the 
measurements already included in the MAT. This should cut the cost of evaluating all PD 
programs across the world. 
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Gathering Data 

Types of measurable data will vary from country to country depending on the level of 
interaction PD officers have with target audiences. The measurement tools should be 
standardized — surveys, interviews, quizzes, evaluations of statements, etc. — but the 
EMU must determine which tool is best for evaluating a program's outcome based on the 
restrictions within the country. The EMU, with the help of the relevant bureau and 
embassy, will also set the minimum and maximum criteria for best and worst outcomes, 
which set baselines for determining when a program is failing or achieving its objective. 

Moving Forward 

The end-user of this model will ultimately decide what is important to the model. The 
project team put great thought into the outcomes, sub-outcomes and measurements; 
however, these are not the only possibilities. If the OMB uses this measurement of PD 
performance as a part of the GPRA, OMB's definitions of what is important will likely 
look different than those developed by DoS or the Office of Policy, Planning and 
Resources. The model presented in this report is not complete, but it is the first step in 
developing a useful tool for measuring PD performance. 

This team hopes that the description of the project and resulting model will be a sufficient 
foundation to enable a subsequent team to continue the project to the next level. While 
great strides have been made in creating a design for a useful model, measuring actions 
that are intertwined with human behavior and perception has proven to be a tremendous 
test. Defining PD programs' strategic goals and identifying which effects are directly 
attributable to PD efforts was a recurring challenge throughout this project. 

Due to time and resource constraints the team was unable to conduct primary field 
research. This type of research will help provide better measurements and a clearer 
connection between the strategic missions for programs, their activities, and their results. 
Therefore, in order to secure a buy-in from the end-user and fill the existing gaps in this 
model, the research team recommends a second team continue this project and engage in 
primary field research. 

The next step is to determine a user. The user or decision maker must then decide their 
priorities for PD. In determining their priorities, the user must decide what to measure. 
The team identified Understanding, Favorability and Influence as important outcomes; 
however, the sub outcomes are still flexible. The team decided that the components 
economy, security, education and environment are important, but the EMU can add or 
replace components they feel are important. Once the EMU or a project team selects the 
measurements for each component, the client should set the data standards and 
definitions as well as how to compare performance. These are relatively simple steps to 
making this model adaptable to the client's goals. 
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Is PD measurable? 

The team's general consensus is that PD is measureable. While quantifying the emotions 
and feelings that PD engenders in recipients and the actions it influences them to take or 
not take is extremely difficult, there are certain proxy measurements that make the task 
feasible. The example measurements included in this report are, in the team's opinion, the 
most accessible and cost-effective to gather. The EMU may decide to add measurements, 
however it should consider the level of resources needed to delve deeper into a PD 
program and its effects. 

Pilot Test 

The team recommends a pilot test of the proposed metrics at select embassies. Certain 
sub-outcomes that have concrete metrics such as trade relationships between businesses 
in the US and other nations, decreased incidences of anti- American public protests, and 
surveys of Fulbright Scholars upon return to their home countries should be the first 
metrics to track in the model. Not only will the pilot provide useful input but it will also 
prove very valuable in securing a buy-in from the eventual users of MAP-PD. 

Some PD programs may need to be altered in the testing phase to examine whether 
changes in programming have an effect, either positive or negative, on the measured 
outcomes. During the pilot and eventually in its full usage, the team believes that the tool 
will be most useful: 

1. For individual PD Officers and PASs to assess PD programs' effectiveness from 
year to year. 

2. As an input to provide guidance to embassy strategic planning efforts. 

3. To enable comparison and feedback among embassies running similar programs 
in different countries. 

Measurement Culture 

Any sustainable attempt to measure or evaluate PD programming will require certain 
structural and cultural changes within the department. Evaluation results are useful guides 
for aiding an organization's self-improvement process. The current evaluation attempts 
within the Office of Public Diplomacy and Public Affairs seem to be a direct result of 
OMB requirements and not a unified organizational priority. Therefore, each department 
works to fulfill the requirements, but does not work with other departments to develop 
best practices or develop uniform analytical tools that can show results across 
departments. This has not only led to the duplication of efforts, but also provides a lack of 
incentive or opportunity for different teams to share ideas. 

Additionally, since each department develops its own measurement methods, the results 
are not uniform or relatable. This makes it difficult for Congress and DoS to allocate 
resources based on priority and success. PD-MAP can be and should be used as a way of 
clarifying strategic missions, highlighting successful efforts, and allocating resources 
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effectively and efficiently. To incorporate this tool appropriately, DoS and OMB need to 
reevaluate their concept of and approach towards measurement. 

Organizations most often use trainings, lectures, and seminars to instigate an 
organizational change towards securing a wider buy-in for an evaluation effort. The team 
recommends that the Office of PD and PA offer and require that senior management and 
certain officers in duty stations to attend evaluation trainings. 

DoS should continue this project; however, it will only produce a useful model with more 
active input from DoS. If a future project team has access to PASs and PD officers in the 
field, they can resolve issues such as metric selection, intertwined outcomes, and 
accounting for the effect of foreign government on PD. Without access to DoS officials, a 
future project will not be able to take the PD-MAP presented in this report and produce a 
complete PD effectiveness measurement system. It is important to understand the work 
presented here is groundbreaking. However, it can only be put to use if this project is 
continued with full DoS support. 
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Appendix A: Survey Questions and Responses 



Respondents 

14 Complete Responses 

I Partial Response (Used - 12 Deleted) 

I I Current DoS Employees 

4 Former DoS Employees and Current Academics 

10 Diplomatic Corps/Foreign Service Officers 

4 Public Diplomacy/Public Affairs Professionals 
7 from United States 
4 from Near East 
1 from Eastern Europe 
1 from Western Europe 
1 from Africa 
1 from South Asia 

Survey of US DoS Public Diplomacy Efforts 

Introduction: 

A graduate student research team at the LB J School of Public Affairs at The University 
of Texas at Austin, in conjunction with the Department of State Office of Analysis, 
Evaluation and Performance Measurement, and the Presidential Advisory Commission on 
Public Diplomacy, is exploring new ways to measure the effectiveness of U.S. Public 
Diplomacy efforts. By answering a few short questions, you will help us understand the 
field of U.S. Public Diplomacy from different perspectives. Your opinion will be 
invaluable in our development of an improved Public Diplomacy measurement system. 
This survey should take no more than 5-10 minutes and will be kept confidential, as it is 
only intended for the purpose of academic research. Thank you for your participation. 

Please select the category that best describes you: 
Current DoS Employee 
Former DoS Employee 

Former DoS Employee and Current Academic 
Academic 
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Please indicate your association with Public Diplomacy in the above capacity: (Check all 
that apply) 

Diplomatic Corps / Foreign Service Officer 
Public Diplomacy / Public Affairs Professional 
Support Staff 
Research 

Academic Faculty 

In what region are you located? 
United States 
Africa 

East Asia / Pacific 
Western Europe 
Eastern Europe 
Near East 
South Asia 

Western Hemisphere (excluding U.S.) 

In your opinion, what is the purpose of US public diplomacy efforts? 

[Text box for up to 500 word response] 

Drawing on your experience in the public diplomacy field, list some long-term (more 
than one year) goals of public diplomacy efforts. 

[Blank text box] 

[Blank text box] 

[Blank text box] 

[Blank text box] 

Drawing on your experience, list some immediate or short-term (shorter than one year) 
goals of public diplomacy efforts. 

[Blank text box] 
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[Blank text box] 
[Blank text box] 
[Blank text box] 

Many public diplomacy professionals suggest the following are long term goals of public 
diplomacy efforts. In order of importance, please rank the following goals for your 
current post. (1 = most important; 5 = least important). Rank the following items using 
numbers from 1 to 5: 

Security 

Cultural Awareness 

Understanding U.S. Policy 

Favorable View of the U.S. 

Strengthening ties with U.S. Foreign Government 

In trying to achieve these long term goals, what are some of the indicators you use to 
assess the effectiveness of public diplomacy efforts, (e.g. anecdotal evidence, newspaper 
articles, media broadcasts, etc.) 

[Blank text box] 
[Blank text box] 
[Blank text box] 
[Blank text box] 

Drawing on your expertise in the PD field, how do you suggest the measurement of 
Public Diplomacy efforts be improved? 

[Text box for up to 500 word response] 

Please provide your email address if you would like to receive a summary copy of the 
results of this survey: 

[Text box for email address] 
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Question 1: In your opinion, what is the purpose of US public diplomacy efforts? 

Al: Explain and demonstrate US policy & values 

A2: Communicate with foreign audiences information and values of the U.S. in terms and 
language and within cultural context that will resonate with the foreign audiences. 

A3: To build relationships with other countries, both with government counterparts and 
with the general populace; to tell the U.S. story and share our culture, values and ideals; 
and to provide opportunities for cultural, scientific and educational exchanges. 

A4: to provide the public in our host country with an objective and cogent explanation of 
US foreign policy, to counter subjective and negative reports by hostile media (often 
state supported by hostile regimes). To provide the public in our host country with 
exposure to the best that American culture has to offer, to counter negative portrayals of 
the united states and American culture presented by American commercial media and 
local media sources. 

A5: Understand the world in which we are operating, engage people from other countries 
so that we can learn about them and share our beliefs, values and information about U.S. 
policy 

A6: To reach a broader audience than traditional government-to-government diplomacy 
using the media (traditional, modern, and emerging) and educational, representational, 
and cultural events and exchanges, with a goal of effectively conveying U.S. values and 
foreign policy messages, and to counter distortions about the U.S. and its policies 
stemming from deliberate and inadvertent mis- and disinformation. There must also be 
an element of "listening" to the reactions from the target audience. 

A7: To give an honest and convincing explanation to foreign audiences, especially the 
elite, about the United States and its policies, and in so doing build support for US goals 
and objectives. 

A8: The clear purpose of US public diplomacy efforts is to enable foreign audiences to 
have a complete and accurate of the U.S.; particularly US foreign policy but also our 
society and culture. We do this by engaging overseas—primarily with elites—but also with 
the broader public. We are not successful at reaching target audiences for a variety of 
reasons, which I am hopeful we will explore. 

A9: 1) To (as the old USIA slogan went) tell America's story to the world. That is, to get 
out accurate information about the United States, its government and people to audiences 
throughout the world. This is done through people-to-people exchange (Fulbright, 
International Visitors, Speakers) as well as through distribution of print, video and 
electronic media, 2) To assist other nations in building their democracies, economies, and 
in confronting social problems through information sharing. 
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A10: To understand, engage, inform, and influence (in that order) foreign publics about 
the United States and U.S. government interests and objectives. 

All: To reach the public, influence public opinion overseas. 

A12: US PD seeks to foster communication with public abroad in order to maximize 
cooperation and minimize conflict with the U.S. This purpose includes not only inter- 
governmental but also inter-societal relations. 

A13: The purpose of US public diplomacy efforts is to advance and explain the policies 
of the United States government to foreign governments, agencies, opinion makers and 
the general public. Those practicing public diplomacy strive to create a bilateral dialogue 
based on mutual respect that recognizes the common values held by different countries 
without minimizing or ignoring the challenges inherent when governments have 
competing claims for primacy, dominance or influence. 

A14: To support U.S. national interests by explaining U.S. policy and American society 
and culture to foreign publics, engaging with them on issues of mutual interest and 
advising US policy makers on foreign opinion to assist in policy formation. 

A15: The purpose of U.S. public diplomacy is to persuade the governments, elites, and 
general public (in that order of importance) to support American positions on major 
international questions. In this sense, public diplomacy is an instrument of national 
power, along with military power, diplomatic power, economic power and intelligence 
power to affect the behavior of other players in global politics to advance and protect 
American interests. 



Question 2: Drawing on your experience in the public diplomacy field, list some long 
term (more than one year) goals of public diplomacy efforts. 

- Make friends for America 

- Multicultural aspect 

- Broaden the public understanding of how freedom of expression works 
Convince recipients that there is a new US foreign policy post Bush 

- Increase awareness by foreign publics about U.S. Policy 

- Increase the number of exchange students in the U.S.; then listen to them 

- Build support for deployment of Missile defense in Poland 
Increase sympathy and understanding of USG foreign policy 

- Mutual Understanding 

- Present (and sometimes promote) U.S. cultural values 

- Improve Image of US overseas 

- Democratic Values 

- Broaden public understanding of how a democracy works 

- Restore US credibility regarding human rights 

- Increase understanding of American values 
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Increase the number of U.S. exchange students abroad; then listen to them 

Explain policies on climate change 

Develop a cadre of professionals familiar with the U.S. 

Support for Democracy 

Promote educational and cultural exchanges to increase mutual understanding 
Gain support for US foreign policies 
Religious tolerance 

Enhance the confidence of girls and women to pursue educational opportunities 

Present the rapid change taking place in US as it adapts to changing world 

Increase international cooperation on major issues facing us 

Listen in general to feedback from foreign audiences; i.e., make PD more 

"interactive" 

Combat corruption and promote the rule of law 
Cultivate successor generations in the Third World. 
Economic Development 

Patiently dispel misconceptions about the U.S. and its policies to enable more 
constructive addressing of issues 

Build bridges of understanding between US + other nations 
Respect for human rights 

English language exchange programs for high school students 

Portray US as a credible partner in multilateral diplomacy 

Improve networking; i.e., develop and use better databases to stay in touch with 

members of the target audience 

Defend US policies in the War on Terrorism 

Develop a more accurate picture of U.S. society & culture. 

Human Rights, broadly defined 

Build better understanding, if not agreement, of U.S. goals 
Supporting emerging democracies in economic development 
Understanding of US policy, society and culture by foreigners 
Promote alignment of nations with U.S. foreign policy goals 
Foster the institutional/organizational/personal ties that help promote 
understanding 

Increasing understanding between the US and adversaries. 
Establishment of personal connections abroad for Americans 
Strengthen the WMD non-proliferation regime 

Enhance support for key strategic commitments of the US, e.g. combating 
terrorism, supporting democratic governance and human rights, promotion of 
environmental cooperation 
Security energy resources 

Understanding of foreign opinion by US policy makers 
Delegitimize the use of terror for political goals 

Work on re-education of younger generation of Israelis and Arabs toward co- 
existence and support for sustainable peace 
Promoting the rule of law and good governance 
Enhancement of educational and professional exchanges 
Promote respect for fundamental human rights 
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Question 3. Drawing on your experience in the public diplomacy field; list some 
short term (less than one year) goals of public diplomacy efforts. 

- Expose foreign publics to our message through coverage of bilateral projects and 
events in the media 

- Re-create USIA; separate the formal PD function from State 

- Publicize the 90 anniversary of US-Polish relations 

- Reach out to younger and female audiences here. 

- Dissemination of White House messaging 

- Explain specific policies 

- Introduce foreign audiences to aspects of US culture 

Select and sponsor future leaders to attend programs and universities in the U.S. 
Give senior PD officers a serious seat at the policy-making table 

- Explain Abu Ghraib 

- Develop contacts outside of our familiar sphere 
Countering false information distributed about the U.S. 

- Refute inaccuracies and falsehoods about the U.S. and its policies 

- Gain support for US foreign policy goals 

- Recruit more PD officers with 4/4 or higher language skills 
Sell a particular weapons system 

Cultivate religious leaders who are very influential 

- Promoting country-specific programs 

- Influence publics to create or advocate for policies in line with U.S interests 

- Respond to negative press/attitudes about US policy 

- Increase the number of USG-funded exchange programs for the poor. 
Show solidarity on the occasion of a Muslim holiday 

- Further efforts to encourage grass-roots civil society 

- Promotion of the DoS's strategic objectives 

- Help maintain public support in Europe on Iranian policies 
Supporting the Iranian democracy movement 

- Understanding of US foreign policy 

- Build broad international public support for steps to persuade Iran and North 
Korea to give up nuclear weapons 

- Help build a more fruitful climate of opinion in Pakistan re. Taliban and al-Qaida 
in Pakistan and Afghanistan 

- Increasing cooperation on counterterrorism internationally 

- Reduction of misunderstandings of US society and culture 

- Persuade Pakistan, Afghanistan and NATO nations to support U.S. goals in the 
Af/Pak strategy 

- Foster concerted programs on energy/climate change in key countries, e.g. Brazil, 
Russia, China, India, Nigeria, Saudi, Iraq, Iran 

- Increasing public/private partnerships in economic development 

- Support for US global interests 

- Ensure that the war on Al-Qaeda is not seen as a Western war on Islam 
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- Engaging youth in social networks toward substantive discussion 

- More open communication globally 

- Persuade elites and publics in BRIC countries to accept limits on carbon 
emissions 



Question 4: Many public diplomacy professionals suggest the following are long 
term goals of public diplomacy efforts. In order of importance, please rank the 
following goals for your current post. (1 = most important; 5 = least important) 

Frequency Table 





One 


Two 


Three 


Four 


Five 


Security 


2 


2 


1 


3 


7 


Cultural Awareness 


2 


3 


4 


1 


5 


Understanding US 
policy 


7 


6 


1 


1 


0 


Favorable View of 
the US 


3 


3 


7 


2 


0 


Strengthening US 
ties with Foreign 
Governments 


1 


1 


5 


5 


3 



Question 5: In trying to achieve these long-term goals, what are some of the 
indicators you use to assess the effectiveness of public diplomacy efforts? (E.g. 
anecdotal evidence, newspaper articles, media broadcasts, etc.) 

- Anecdotal evidence 

- Audiences ratings during media broadcast 

- Reports from public diplomacy professionals based abroad 

- Readiness of other governments to align themselves with U.S. policy 

- Media commentary and coverage 
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Opinion makers analysis of US policies 
Anecdotal evidence from US diplomats 

Reduction of public pressure on elites in foreign countries to distance the country 
from U.S. objectives 
Public opinion polls 

Public polls moving toward support of US Policy goals (not popularity of US) 
Newspaper articles 

Reduction in tendency of political parties to "campaign" against the U.S. in 
elections 

Focus group discussions 

# of collaborative projects with experts in various fields 
Broadcasts 

Ranking of U.S. in public opinion polling 

Opinion leadership interviews 

Receptiveness of leaders to support US policies 

Information on the Internet 

Polling 

Media tone and content 
Newspaper articles 

Positive news coverage of USG activities 

Behavior of those who have participated in International Visitor Leadership and 

other exchange programs 

Surveys 

Contacts 

Foreign government public statements 
Media broadcasts 

Audience reaction at USG hosted and organized events 
Polling with good controls before and after PD efforts 
Press articles 
Media coverage 
Government statements 
Polling data 

Broadcast media programs 

Breakthroughs in diplomatic negotiations/international agreements 

Newspaper articles 

Political party statements 

Blog comments 

Local staff 

Polling data 

Anecdotal evidence 

Application rates for USG exchanges and grants programs 

Number of favorable op-eds, blogs, editorials, etc. by influencers in host countries 
Political support within US for foreign policy initiatives promoted through public 
diplomacy overseas 
On-line commentaries 
Demonstrations 
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- Activities of alumni of USG sponsored exchange programs 

- Anecdotes 

- Key leader attitudes 
Speaker programs 

One-on-one discussion with host country nationals 

- Funding approved by Congress for public diplomacy efforts 

- Interest in participating in exchange programs 
Statements by contacts 

- Media 

- Websites/Facebook—electronic and social media 



Question 6. Drawing on your expertise in the PD field, how do you suggest the 
measurement of Public Diplomacy efforts be improved? 

Al: Less interaction with institutions. More interaction with people. 

A2: Focus on government actions, rather than pronouncements and media reports. Set 
strict criteria for cultural programs - make sure that precious resources are spent wisely - 
make sure that only the very best cultural offerings are presented to the host country 
public make ambassadorial support the key function of the PD section, ensure that the 
ambassador provides plenty of feedback regarding this support, especially in the area of 
press relations, speeches and public appearances, coordinate closely with the economic 
and policy sections 

A3: We need to identify good short term indicators that show evidence of leading to 
long-term results, and benchmarks that can show trends. 

A4: Try to be as quantitative as possible, but always remember PD is more art than 
science. We must redouble our efforts to reach people "on the street." We must get away 
from the fear and "security uber alles" mindset we have developed, especially post- 9/1 1. 
We must not be foolish, but we must also be more open and accessible to everyday 
people in the countries where we serve. American Presence Posts and American Corners 
are programs that should be carefully studied, improved, and expanded. 

A5: PD officers in the field have to be more nimble in ensuring that there is a response, if 
one is needed, with the 24-your news cycle. Two or more days later, is too late to affect 
public attitudes. 

A6: We need to get far less quantitative. Far more attention must be paid to long-range 
efforts. Please try to measure effectiveness over a longer time-span. We are making 
investments that should pay dividends in 20 years. 

A7: There needs to be more of an effort made to cover long-range effects. Current 
measurements efforts are extremely short term. The overall impact of an exchange, 
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speaker, material donation, etc. may not be evident for many years. Institutional memory 
is short, and many of the more subtle long term effects are lost. 

A8: More focus on the content of media coverage rather than a compilation or list of 
"media placements." Continued follow-up via social networks with exchange visitors and 
audience members of speaker and cultural exchange programs. 

A9: Don't know, not a PD expert. 

A10: Anecdotal evidence is most useful; limited, realistic goals and assessing the 
delivery of PD programs, information, messages are possible. Long-term measurements 
of exchange results quite possible. Don't try to either credit or blame PD for the public in 
country X agreeing with US policy. There are too many intervening forces totally out of 
our control. 

Al 1: Focusing on each country individually and executing a comprehensive strategy for 
each country that includes social media, TV and print. Make a targeted effort over a 
specific time. Polling data from before and after that targeted effort, focusing on the 
specific policy perception to be improved, would yield success or failure by degrees. 
Methods shown to work should be kept, and those less effective discarded. Increase use 
of ambassadors, FSOs and LES in tracking PD efforts. Worked in Nordics. 

A12: Increase media reaction reports from all embassies, including analysis by trained 
public diplomacy officers, to include their insights from personal contacts. Strengthen the 
direct connections between PD professionals at embassies and PD professionals at State 
in Washington, and give both of them instructions to enhance reporting on foreign 
opinion trends to policy makers and senior officials at State. 

A13: The real test is whether elites who control policy feel that they are under constant 
public pressure to distance them from key US. objectives. It does not matter much 
whether the U.S. is "popular" with local publics. It matters a lot whether the elites who 
control governments see it in their political interest to align themselves with the U.S. 
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Appendix B: Department of State Public Diplomacy Logic Model 
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Appendix C: Example Public Diplomacy Metrics 



This appendix contains example measurements for use in the PD-MAP. The metrics are 
broken down first by outcome then by audience, policy type, and finally sub-outcomes as 
explained in Chapter 3. These metrics are meant to be suggestions as to the type of 
information that can be used with PD-MAP and they can be replaced in part or as a 
whole. 

In the appendix, each outcome is in its own section. Following the outcome is a table 
showing the outcome breakdown down to the metric name. Following the table is a 
detailed breakdown of each metric in the table including a description of how to calculate 
the metric and examples of ideal performance and minimally acceptable performance. 
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Audit Trail 
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FOREIGN GOVERNMENT 
FOREIGN POLICY 

Understanding -Foreign Government - Foreign Policy- Dissemination (UFGFPD) 

UFGFPD Events Growth 

■ Measurement : Relative change in the number of US policy-related informational 
events directed toward foreign governments in a FY 

■ Unit of Measure : Percent change 

■ Calculation : The number of US policy-related events directed at members of 
foreign governments in a FY minus, the number in the previous FY divided by the 
number in the previous FY 

■ Example : 50% increase in the number of US policy-related info sessions from 
FY09toFY10 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 

UFGFPD Exchange Growth 

■ Measurement : Relative change in the number of positions available for government 
officials in exchanges from the previous FY 

■ Unit of Measure : Percent change 

■ Calculation : The number of exchange opportunities available to government 
officials this FY minus the number of exchange opportunities in the previous FY 
divided by the number available the previous FY 

■ Example : 100 percent increase in the number of exchange opportunities available to 
foreign governments in the International Visitors Leadership Program from FY08 to 
FY09 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 

Understanding- Foreign Government- Foreign Policy - Reception (UFGFPR) 

UFGFPR Events Attend 

■ Measurement : Relative change in the number of government officials who attend 
policy related PD event from the number of government officials who attend 
similar policy related PD event the previous FY 

■ Unit of Measure : Percent change 
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■ Calculation : The number of government officials attending policy related PD events 
in a FY minus the number of government attendees at similar PD event in the 
previous FY divided by the number of attendees in the previous FY 

■ Example : 60 percent increase in number of government officials that attend policy 
info sessions from FY08 to FY09 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 

UFGFPR Exchange 

■ Measurement : Relative change in number of government officials who participate 
in government exchange programs in a FY from the previous FY 

■ Unit of Measure : Percent change 

■ Calculation : The number of government officials participating in official policy 
exchanges minus the number who participated in the previous FY divided by the 
number in the previous FY 

■ Example : 20 percent increase in the number of exchange participants from FY 08- 
FY 09 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 

UFGFPR Media Read 

■ Measurement : Relative change in the number of reports and publications produced 
by US government-produced (e.g. CRS, GAO, OMB) reports and analyses 
requested by the target government 

■ Unit of Measure : Percent change 

■ Calculation : The number of US government produced reports requested by the 
target government minus the number from the previous FY divided by the number 
in the previous FY 

■ Example : 20 percent increase in the number of US reports requested by the target 
government from FY 08- FY 09 

■ Ideal Performance : 50% increase 

■ Minimally Acceptable Performance : 0% increase 



Understanding- Foreign Government- Foreign Policy - Comprehension (UFGFPC) 
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UFGFPC Survey 

■ Measurement : Relative change in number of government officials who correctly 
identify more than 50% of the tenets of presented US policies before and after an 
informational session in pre- and post-event surveys 

■ Unit of Measure : Percent change 

■ Calculation : The number of completed surveys that correctly identify more the 50% 
of US policy tenets after an event minus the number that correctly identify more 
than 50% prior to the event divided by the number correctly identifying more than 
50% prior to the event 

■ Example : 20 percent increase in the number of surveys correctly identifying more 
than 50 percent of US policy tenets after an informational event than prior to it 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 

UFGFPC Statements 

■ Measurement : Relative change in the number of statements (official statements, 
editorials, interviews or blogs) released by government officials after an 
informational event that correctly characterize US policy than prior to the event 

■ Unit of Measure : Percent change 

■ Calculation : The number of statements by foreign government officials correctly 
characterizing US policy after an informational event minus the number of 
statements correctly characterizing US policy prior to an informational event 
divided by the number of prior to the event 

■ Example : 25 percent increase in the number of statements correctly identifying the 
US missile defense program after an informational event 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 

UFGFPC Bad Stmts 

■ Measurement : Relative change in the number of statements (official statements, 
editorials, interviews or blogs) released by government officials after an 
informational event that do not correctly characterize US policy 

■ Unit of Measure : Percent change 

■ Calculation : The number of statements by foreign government officials that do not 
correctly characterize US policy after an informational event minus the number of 
statements not correctly characterizing US policy prior to an informational event 
divided by the number of prior to the event 
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■ Example : 25 percent increase in the number of statements not correctly identifying 
the US missile defense program after an informational event 

■ Ideal Performance : 0% increase 

■ Minimally Acceptable Performance : 50% increase 

CULTURE 

Understanding- Foreign Government- Culture - Dissemination (UFGCD) 
UFGCD Events Grwth 

■ Measurement : Relative change in the number of invitations to members of a foreign 
government to a US cultural event in a FY from the previous FY 

■ Unit of Measure : Percent change 

■ Calculation : The number of invitations extended to members of a foreign 
government to US cultural events in a FY minus the number of invitations extended 
in the previous FY divided by the number of invitations in the previous FY 

■ Example : 15 percent increase in the number of invitations extended from FY08 to 
FY09 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 

UFGCD Exchange Grwth 

■ Measurement : The percent change in the number of positions available for 
government officials in culture related exchanges from the previous FY 

■ Unit of Measure : Percent change over previous FY 

■ Calculation : The number of positions available to government officials this FY - 
the number of positions available the previous FY divided by the number of 
positions available the previous FY 

■ Example : Percent change in the number of slots available to foreign governments in 
cultural exchanges from FY08 to FY09 

■ Ideal Performance : 100% increase 

■ Minimally Acceptable Performance : 50% decrease 
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UFGCD Media Outlet Growth 

■ Measurement : Number change in the US produced public mass-market publications 
requested by, or subscribed to by, foreign government offices 

■ Unit of Measure : Number count 

■ Calculation : The difference in the number of US produced public mass-market 
publication requested by or subscribed to by foreign government officials and 
offices from the previous FY 

■ Example : The number of US publications subscribed to by government officials in 
Ghana plus the number of requests for US mass-market publications from their 
offices this year minus those from the previous FY 

■ Ideal Performance : 20 subscriptions/requests 

■ Minimally Acceptable Performance : 5 subscriptions/ requests 

Understanding- Foreign Government - Culture - Reception (UFGCR) 
UFGCR Events Attend 

■ Measurement : Relative change in the number of foreign government officials who 
attend a US cultural event in one FY from the previous FY 

■ Unit of Measure : Percent change 

■ Calculation : The number of foreign government officials who attend a cultural 
event in a FY minus the number of foreign government officials who attended a US 
cultural event in the previous FY divided by the number in the previous FY 

■ Example : A 45% increase in the number of foreign government officials that 
attended the Ambassador's Fourth of July celebration from FY08 to FY09 

■ Ideal Performance : 50% increase 

■ Minimally Acceptable Performance : 0% increase 

UFGCR Exchange 

■ Measurement : The percent change in the number of government officials 
participating in culture related exchanges from the previous FY 

■ Unit of Measure : Percent change over previous FY 
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■ Calculation : The number of government officials participating this FY - the number 
participating the previous FY divided by the number participating the previous FY 

■ Example : Percent change in the number of slots available to foreign governments in 
cultural exchanges from FY08 to FY09 

■ Ideal Performance : 50% increase 

■ Minimally Acceptable Performance : 0% increase 



Understanding- Foreign Government - Culture - Comprehension (UFGCC) 
UFGCC Survey 

■ Measurement : Relative change in the number of exit survey responses that 
demonstrate comprehension of US culture following a cultural event from one FY 
to another 

■ Unit of Measure : Percent change 

■ Calculation : The number of correct government officials correctly responding to a 
survey following a US cultural event in FY1 minus the number of correct 
government officials correctly responding to a survey following a US cultural event 
in FYO divided by the number of correct government officials correctly responding 
to a survey following a US cultural event in FYO 

■ Example : A 30% increase in the number of foreign government officials who 
correctly identify 3 tenants of what the Fourth of July means to Americans from 
FY08toFY09 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 

UFGCC Statements 

■ Measurement : The number of statements (official statements, editorials, interviews 
or blogs) released by government officials in a FY that correctly characterize US 
culture 

■ Unit of Measure : Number count 

■ Calculation : The number of statements released by foreign government officials 
correctly characterizing US culture in a FY 
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■ Example : The number of statements released by government officials correctly 
describing US counter-culture 

■ Ideal Performance : 150 statements 

■ Minimally Acceptable Performance : 50 statements 

ELITES 

Understanding - Elites - Foreign Policy - Dissemination (UEFPD) 

UEFPD Info Sessions 

■ Measurement : Relative change in the number of US policy-related informational 
events directed toward elites in a FY over the number in the previous FY 

■ Unit of Measure : Percent change 

■ Calculation : The number of US policy-related events directed at elites in a FY 
minus the number in the previous FY divided by the number in the previous FY 

■ Example : 50 percent increase in the number of US policy-related info sessions from 
FY09 to FY10 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 

UEFPD Exchange Grwth 

■ Measurement : The relative change in number of spaces for elites to participate in 
exchange programs from the previous FY 

■ Unit of Measure : Percent change 

■ Calculation : The number of spaces for elites participating in policy-related 
exchanges minus the number participating in the previous FY divided by the 
number from the previous FY 

■ Example : Percent change in the number of spaces for Fulbright participants from 
FY 08- FY 09 

■ Ideal Performance : 100% increase 

■ Minimally Acceptable Performance : 50% increase 

Understanding- Elites — Foreign Policy - Reception (UEFPR) 
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UEFPR Info Attendees 

■ Measurement : Relative change in the number of elites who attend policy related PD 
event from the number of elites who attend similar policy related PD event the 
previous FY 

■ Unit of Measure : Percent change 

■ Calculation : The number of elites attending policy related PD events in a FY minus 
the number of elites at similar PD event in the previous FY divided by the number 
of elites attending in the previous FY 

■ Example : 60 percent increase in number of government officials that attend policy 
info sessions from FY08 to FY09 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 

UEFPR Exchange 

■ Measurement : The relative change in number of elites participating in exchange 
programs from the previous FY 

■ Unit of Measure : Percent change 

■ Calculation : The number of elites participating in policy-related exchanges minus 
the number participating in the previous FY divided by the number from the 
previous FY 

■ Example : Percent change in the number of Fulbright participants from FY 08- FY 
09 

■ Ideal Performance : 50% increase 

■ Minimally Acceptable Performance : 0% increase 

Understanding- Elites- Foreign Policy - Comprehension (UEFPC) 
UEFPC Editorials 

■ Measurement : Relative change in the number of editorials released by elites after an 
informational event that correctly characterize US policy than prior to the event 

■ Unit of Measure : Percent change 

■ Calculation : The number of editorials by elites correctly characterizing US policy 
after an informational event minus the number of editorials correctly characterizing 
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US policy prior to an informational event divided by the number of prior to the 
event 

■ Example : 25 percent increase in the number of editorials correctly identifying the 
US missile defense program after an informational event 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 

UEFPC Articles 

■ Measurement: Relative change in the number of articles correctly representing US 
policy in the major publications of the country 

■ Unit of Measure: Percent change 

■ Calculation: Change in the number of articles correctly delineating US policy. Take 
the total number of articles correctly identifying US policy in the current year minus 
the previous year, and divide by the total in the current year 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 

UEFPC Blog Hits 

■ Measurement: Relative change in the number of blog posts correctly representing 
US policy in the major blog publications of the country 

■ Unit of Measure: Percent change 

■ Calculation: Change in the number of blog posts correctly delineating US policy. 
Take the total number of blog posts correctly identifying US policy in the current 
year minus the previous year, and divide by the total in the current year. Multiply 
the resulting number by 100 to get a percent. 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 

CULTURE 

Understanding- Elites - Culture - Dissemination (UECD) 
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UECD survey 

■ Measurement : Relative change in percent of survey respondents who are familiar 
with the US produced publications on the survey list that are circulated/available 
in the host country 

■ Unit of Measure : Percent change 

■ Calculation : Percent familiar with US publication this FY minus the previous. 
Divide by the percent familiar with US publications from the previous FY 

■ Example : A 30% increase in the percentage of respondents familiar with US 
produced publications 

■ Ideal Performance : 50% increase 

■ Minimally Acceptable Performance : 50% decrease 

UECD Exchange Grwth 

■ Measurement : Relative change in the number exchanges that occur this year 
compared to last FY 

■ Unit of Measure : Percent change 

■ Calculation : take the number of exchanges that occurred this year divided by the 
number that occurred last year times 100 

■ Ideal Performance : 100% increase 

■ Minimally Acceptable Performance : 50% increase 

Understanding - Culture - Elites - Reception (UCER) 
UECR Exchange Applications 

■ Measurement : Relative change in the number of exchange program participant 
applications from one FY to another 

■ Unit of Measure : Percent change 

■ Calculation : The number of exchange program participant applications in a FY 
minus the number in the previous FY divided by the number in the previous FY 

■ Example : A 30% increase in the number of exchange program participant 
applications from FY08 to FY09 
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■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 

UECR Media Responses 

■ Measurement : Relative change in number of embassy representatives' media 
interviews from the previous year. 

■ Unit of Measure : Percent change. 

■ Calculation : The change in the number of media interviews (radio, television, and 
newspaper interviews in which embassy representatives explain US policy) from 
those in the previous FY year divided by the number in the previous year. 

■ Example : 50 interviews in 2009. 100 interviews in 2008. (50-100)/100 = 50% 
decrease. 

■ Ideal Performance : 50% increase 

■ Minimally Acceptable Performance : 0% increase 

UECR Fulbrights 

■ Measurement: Relative change in Fulbright scholars' knowledge of the US after 
Fulbright exchange 

■ Unit of Measure: Percent change 

■ Calculation: Utilize the surveys Fulbright scholars complete on their return to their 
home country versus surveys they completed prior to beginning the Fulbright 
program, compared to the previous year's data. 

■ Ideal Performance : 50% increase 

■ Minimally Acceptable Performance : 0% change 

Understanding- Elites- Culture - Comprehension (UECC) 
UECC Survey 

■ Measurement : Overall score of survey respondents concerning the improvement of 
their understanding of the roots of US culture and mores 
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■ Unit of Measure : Number count 

■ Calculation : Likert scale score of respondents 

■ Example : The previous FY, those surveyed gave an average response of 5 meaning 
they had a basic understanding of US culture but much improvement remained to be 
made. 

■ Ideal Performance : 10 

■ Minimally Acceptable Performance : 0 

UECC Statements 

■ Measurement : The number of statements (editorials, interviews or blogs) released 
by elites in a FY that correctly characterize US culture 

■ Unit of Measure : Number count 

■ Calculation : The number of statements made by elites correctly characterizing US 
culture in a FY 

■ Example : The number of statements released by elites correctly describing US 
counter-culture 

■ Ideal Performance : 150 statements 

■ Minimally Acceptable Performance : 50 statements 

UECC Focus Group 

■ Measurement : Overall score of focus group respondents concerning the 
improvement of their understanding of the roots of US culture and mores 

■ Unit of Measure : Number count 

■ Calculation : Likert scale score of respondents 

■ Example : The previous FY, those in the focus group gave an average response of 5 
meaning they had a basic understanding of US culture but much improvement 
remained to be made. 
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Ideal Performance : 10 

Minimally Acceptable Performance : 0 



GENERAL POPULATION 

Understanding- General Population- Foreign Policy - Dissemination (UGPFPD) 

UGPFPD Press Briefings 

■ Measurement : Relative change in number of press briefs released by the embassy 
from the previous year. 

■ Unit of Measure : Percent change 

■ Calculation : The change in the number of press briefs/releases (statements issued by 
embassy staff that informs the press and the general population of US policy and 
events involving the US) provided from the previous FY divided by the numbers of 
press briefs/releases provided in the previous FY. 

■ Example : 200 press briefs in 2009. 150 press briefs in 2008. +50 press briefs. 33% 
increase. 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 

UGPFPD Media Interviews 

■ Measurement : Relative change in number of embassy representatives' media 
interviews from the previous year. 

■ Unit of Measure : Percent change 

■ Calculation : The change in the number of media interviews (radio, television, and 
newspaper interviews in which embassy representatives explain US policy) from 
those in the previous FY year divided by the number in the previous year. 

■ Example : 50 interviews in 2009. 100 interviews in 2008. (50-100)/100 = 50% 
decrease. 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 
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UGPFPD Policy Workshop 

■ Measurement : Relative change in policy- related educational workshops from the 
previous year. 

■ Unit of Measure : Percent change from the previous FY. 

■ Calculation : The change in number of educational workshops (workshops for 
national teachers where US policy is taught and materials for the purpose of 
teachers educating their students on US policy) from the previous FY divided by the 
number of educational workshops in the previous FY. 

■ Example . 15 workshops in 2009. 10 workshops in 2008. (15-10)/10 = 50% increase. 

■ Ideal Performance : 50% increase. 

■ Minimally Acceptable Performance : 0% increase 

Understanding — General Population — Foreign Policy — Reception (UGPFPR) 
UGPFPR Press Briefings 

■ Measurement : Relative change in attendance at US conducted Policy Briefings by 
host country "elites" 

■ Unit of Measure : Percent change 

■ Calculation : The change in number of elites that attend US conducted policy 
briefings from the previous FY divided by the number of elite attendees in the 
previous FY. 

■ Example . 15 elites attended policy briefings in FY09. 10 attended in FY08. (15- 
10)/10 = 50% increase. 

■ Ideal Performance : 100% increase. 

■ Minimally Acceptable Performance : 10% increase 

UGPFPR Media Interview 

■ Measurement : Relative change in "Favorable" interviews of host-country elites 
about US foreign policy 

■ Unit of Measure : Percent change 



90 



■ Calculation : The change in number of interviews of host-country elites in which 
they give "favorable" reviews or assessments of US policy from the previous FY 
divided by the number in the previous FY. 

■ Example . 6 elites gave favorable assessments of US policy in interviews in FY09. 2 
gave favorable assessments in FY08. (6-2)/2 = 200% increase. 

■ Ideal Performance : 50% increase. 

■ Minimally Acceptable Performance : 0% increase 

UGPFPR Policy Workshop 

■ Measurement : Relative change in participation in policy- related educational 
workshops from the previous year 

■ Unit of Measure : Percent change 

■ Calculation : The change in number of participants in educational workshops 
(workshops for national teachers where US policy is taught and materials for the 
purpose of teachers educating their students on US policy) from the previous FY 
divided by the number of participants in educational workshops in the previous FY. 

■ Example . 15 participants in workshops in 2009. 10 participants in 2008. (15-10)/10 
= 50% increase. 

■ Ideal Performance : 50% increase. 

■ Minimally Acceptable Performance : 0% increase 

Understanding— General Population — Foreign Policy — Comprehension (UGPFPC) 
UGPFPC Survey 

■ Measurement : Score of survey respondents about having a better understanding of 
the US policy objectives in the survey 

■ Unit of Measure : Score 

■ Calculation : Likert scale score of respondents 

■ Example : The previous FY, those surveyed gave an average response of 5 meaning 
they had a basic understanding of US culture but much improvement remained to be 
made. 

■ Ideal Performance : 10 



91 



Minimally Acceptable Performance : 0 



UGPFPC Statements 

■ Measurement : Relative change in the number of "favorable" responses from "man- 
on-the-street" questions about a US policy 

■ Unit of Measure : Percent change 

■ Calculation : The percentage of "man-on-the-street" interviews with favorable views 
of US policy minus the percentage from the previous FY, divided by the previous 
FY 

■ Example : 25 percent of Egyptian "men-on-the-street" viewed US policy favorably 
in FY08 and 50 percent in FY09, representing a 100 percent increase 

■ Ideal Performance : 150% increase 

■ Minimally Acceptable Performance : 50% decrease 

UGPFPC Focus Groups 

■ Measurement : Overall score of general population focus group respondents 
concerning the improvement of their understanding of US policy 

■ Unit of Measure : Number count 

■ Calculation : Likert scale score of respondents 

■ Example : The previous FY, those in the focus group gave an average response of 5 
meaning they had a basic understanding of US policy but much improvement 
remained to be made. 

■ Ideal Performance : 10 

■ Minimally Acceptable Performance : 0 

CULTURE 

Understanding - General Population -- Culture - Dissemination (UGPCD) 
UGPCD Culture Events 
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■ Measurement : Relative change in number of American cultural events hosted by the 
embassy from the previous year. 

■ Unit of Measure : Percentage change 

■ Calculation : The change in number of American cultural events (music shows, 
author book reading events, art shows, athlete speaker events, etc., performed in the 
host country) in a FY from the number in the previous FY divided by the number in 
the previous FY. 

■ Example : 200 events in 2009. 150 events in 2008. (200-150)/150 = 33% increase. 

■ Ideal Performance : 100% increase 

■ Minimally Acceptable Performance : 10% increase 



UGPCD Entertain Media 

■ Measurement : Relative change in number of television and radio shows that present 
US culture from the previous year 

■ Unit of Measure : Percentage change 

■ Calculation : The change in number of television and radio shows (shows arranged 
by the embassy to present American cultural guests or acts, give interviews about 
American life and culture, present stories or video of American culture, etc.) in a 
FY from the number of shows in the previous FY divided by the number in the 
previous FY. 

■ Example : 100 shows in 2009. 110 shows in 2008. (100-1 10)/1 10 = 9% decrease. 

■ Ideal Performance : 100% increase 

■ Minimally Acceptable Performance : 5% increase 



UGPCD School Visits 

■ Measurement : Number of school visits for the purposes of presenting US culture 
from the previous year 

■ Unit of Measure : Number change 
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■ Calculation : The change in the number of school visits (visits to schools by 
embassy representatives or visits of non-embassy staff, such as an artist or author, 
arranged by the embassy to present US culture) from the number of visits in the 
previous FY divided by the number in the previous FY. 

■ Example : 150 visits in 2009. 100 visits in 2008. (150-100)/100 = 50% increase. 

■ Ideal Performance : 20 visit increase 

■ Minimally Acceptable Performance : 5 visit increase 

Understanding— General Population — Culture — Reception (UGPCR) 
UGPCR Events Attended 

■ Measurement : Relative change in number of attendees of American cultural events 
targeted to the general population attended from the previous year 

■ Unit of Measure : Percentage change 

■ Calculation : The change in number of attendees to American cultural events (music 
shows, author book reading events, art shows, athlete speaker events, etc., 
performed in the host country) in a FY from the number in the previous FY divided 
by the number in the previous FY. 

■ Example : 200 events in 2009. 150 events in 2008. (200-150)/150 = 33% increase. 

■ Ideal Performance : 100% increase 

■ Minimally Acceptable Performance : 10% increase 



UGPCR Media Circulation 

■ Measurement : Relative change in national subscription rates for US produced 
"mass media" publications (newspapers, magazines, etc.) 

■ Unit of Measure : Percent change 

■ Calculation : The number of US produced "mass media" publication subscriptions 
by the target country minus the number from the previous FY divided by the 
number in the previous FY 

■ Example : 20 percent increase in the number of US produced "mass media" 
publication subscriptions by the target country from FY 08- FY 09 

■ Ideal Performance : 100% increase 
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■ Minimally Acceptable Performance : 50% decrease 

UGPCR School Visit Attended 

■ Measurement : Relative change in number of attendees to school visits for the 
purposes of presenting US culture from the previous year. 

■ Unit of Measure : Percent change 

■ Calculation : The change in the number of attendees to school visits (visits to 
schools by embassy representatives or visits of non-embassy staff, such as an artist 
or author, arranged by the embassy to present US culture) from the number of visit 
attendees in the previous FY divided by the number in the previous FY. 

■ Example : 150 visits in 2009. 100 visits in 2008. (150-100)/100 = 50% increase. 

■ Ideal Performance : 50% increase 

■ Minimally Acceptable Performance : 0% increase 

Understanding — General Population — Culture — Comprehension (UGPCC) 
UGPCC Survey 

■ Measurement : Score of survey respondents about having a better understanding of 
the US culture and mores in the survey 

■ Unit of Measure : Score 

■ Calculation : Likert scale score of respondents 

■ Example : The previous FY, those surveyed gave an average response of 5 meaning 
they had a basic understanding of US culture but much improvement remained to be 
made. 

■ Ideal Performance : 10 

■ Minimally Acceptable Performance : 0 

UGPCC Statements 

■ Measurement : Relative change in the number of "favorable" responses from "man- 
on-the-street" questions about a US culture and mores 
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■ Unit of Measure : Percent change 

■ Calculation : The percentage of "man-on-the-street" interviews with favorable views 
of US culture minus the percentage from the previous FY, divided by the previous 
FY 

■ Example : 25 percent of Egyptian "men-on-the-street" viewed US culture favorably 
in FY08 and 50 percent in FY09, representing a 100 percent increase 

■ Ideal Performance : 100% increase 

■ Minimally Acceptable Performance : 0% increase 

UGPCC Focus Groups 

■ Measurement : Overall score of general population focus group respondents 
concerning the improvement of their understanding of US culture and mores 

■ Unit of Measure : Score 

■ Calculation : Likert scale score of respondents 

■ Example : The previous FY, those in the focus group gave an average response of 5 
meaning they had a basic understanding of US culture and mores but much 
improvement remained to be made. 

■ Ideal Performance : 10 

■ Minimally Acceptable Performance : 0 
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Section II: Favor ability 
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FOREIGN GOVERNMENT 
POLICY 

Favorability — Foreign Government — Policy— Security (FFGPS) 
FFGPS NPT Adherence 

■ Definition : Determination of whether or not a country is adhering to their NPT 
agreement; and if they have not signed the NPT, whether or not the country is 
adhering to security requirements set forth by the US 

■ Unit of Measure : A scale of 1 to 10 

■ Calculation : 1 would be no, 5 being partially, and 10 being yes. 

■ Example : Iran would be a 1 because they have violated the NPT. Russia is 
expanding their nuclear capabilities so it would receive a 5. Japan is adhering to the 
NPT and would receive a 10. 

■ Ideal Performance : 10 

■ Minimally Acceptable Performance : 1 

FFGPS Official Protests 

■ Definition : Relative change in the number of official protests relayed by the country 
to the US embassy regarding domestic security policies in the US 

■ Unit of Measure : Percent change 

■ Calculation : The number of official protests registered. Summoning the 
ambassador/ commissioner should also be treated as an official protest. 

■ Example : In 2006 the Pakistani government released 3 official protests against 
racial profiling of its citizens. 

■ Ideal Performance : 100% decrease 

■ Minimally Acceptable Performance : Any Increase 

Favorability — Foreign Government — Policy — Economic (FFGPE) 
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FFGPE WTO/UN Voting 



■ Definition : Relative change in the number of new or modified trade agreement that 
is made coinciding with US policy interests or UN vote that is made coinciding 
with US policy interests shows Favorability. 

■ Unit of Measure : Percent change 

■ Calculation : Add the number of favorable trade agreements and UN votes and 
divide that number by the total number of trade agreements and UN voting issues 
during that fiscal year and then calculate the percentage of favorable 

votes/ agreements . 

■ Example : India signed 3 new trade agreements coinciding with US policy interests 
and voted 8 times with US voting in the UN out of 10 trade agreements. 50 UN 
issues were voted on (3+8=11; (11/60) x 100 =18.3%). 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 



FFGPE Red Tape 

■ Definition : Relative change in the length of time, on average, it takes a company to 
be allowed to do business within the country (in days) from previous FY 

■ Unit of Measure : Percent change 

■ Calculation : Percent change in the length of time, on average, it takes a company to 
be allowed to do business within the country (in days) from previous FY 

■ Example : The number of days it takes for the Russian government or Chinese 
government to approve an American company, such as Wal-Mart, to do business in 
the aforementioned countries is much longer than France or Canada. 

■ Ideal Performance : 125% improvement 

■ Minimally Acceptable Performance : 25% increase 



CULTURE 

Favorability — Foreign Government — Culture (FFGC) 
FFGC Civil Liberties 
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■ Definition : The Freedom House report of Civil Rights ratings 

■ Unit of Measure : Score 

■ Calculation : Determined by Freedom House report methodology. 

■ Example : Between January 1 st 2008 and December 31 st 2008 Afghanistan's Civil 
liberties improved from a 7 to a 6. 

■ Ideal Performance : Civil Liberties index score = 1 

■ Minimally Acceptable Performance : Civil Liberties index score = 7 

ELITES 
POLICY 

Favorability— Elites— Policy— Security (FEPS) 
FEPS Visas 

■ Definition : Relative change in the number of US visas sought by the elite shows 
interest and the comfort level of foreigners traveling into the United States. 

■ Unit of Measure : Percent change 

■ Calculation : The difference between the numbers of visa applications from this year 
by the previous year divided by visas sought the previous FY. 

■ Example : If 500 business owners and religious leaders (among other distinguishable 
elite) applied this year and only 100 applied last year. (500% increase). 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 

FEPS Protection 

■ Definition : Relative change from year to year of policies be it at hotels, airports, 
public buildings, that protect elite travelers. 

■ Unit of Measure : Percent change 
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■ Calculation : Percent change from year to year of policies, be it at hotels, airports, 
public buildings, that protect elite travelers. 

■ Example : Hotels in countries with high levels of theft should perform background 
checks on hotel employees to lower the level of theft within hotels 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 

Favorability— Elites— Policy— Economic (FEPE) 
FEPE Partnerships 

■ Definition : Relative change in business partnerships between US companies and 
companies within a country 

■ Unit of Measure : Percent change 

■ Calculation : An increase in business partnerships between US companies and 
companies within a country shows favorability for US businesses and products. 

■ Example : In Dubai US companies cannot operate unless they have a local company 
sponsor them. 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 

FEPE Fin'l Incentives 

■ Definition : Relative change in the dollar amount of money from the foreign country 
to the US invested from previous FY 

■ Unit of Measure : Percent change 

■ Calculation : The difference between the dollar amount of money ($) invested this 
year and the dollar amount of money ($) invested last year divided by the dollar 
amount of money ($) last year multiplied by a hundred. 

■ Example : $2.5 million was invested in the US companies during the current fiscal 
year. $1.5 million was invested in US companies during the previous fiscal year. 

■ Ideal Performance : 125% increase 
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■ Minimally Acceptable Performance : 25% decrease 
CULTURE 

Favorability— Elites— Culture (FEC) 
FEC Advertising 

■ Definition : Relative change in advertising sold by foreign companies (i.e. 
billboards, commercials) to US businesses 

■ Unit of Measure : Percent change 

■ Calculation : Difference between current sales and previous year's sales multiplied 
by a hundred. 

■ Example : A marketing company advertises an American singer on a billboard in 
order to increase the number of listeners who purchase that album. 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 

FEC Tourism Opportun's 

■ Definition : Relative change in number of laws or amendments made that encourage 
the wealthy to vacation within a certain country 

■ Unit of Measure : Percent change 

■ Calculation : The numbers of laws/amendments passed that encourage tourism 
minus the same the previous year divided by the number of laws/amendments the 
previous year. 

■ Example : The UAE passes tourist policies to ensure that American tourists will stay 
at resorts such as the Atlantis. 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 



FEC US Media Sales 
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■ Definition : Relative change in the number of American movies sold in a FY and the 
number of American movies sold the previous FY 

■ Unit of Measure : Percent change 

■ Calculation : The difference between the number of American movies sold in a FY 
and the number of American movies sold the previous FY divided by the number of 
American movies sold last year multiplied by a hundred. 

■ Example : A country sells 800,000 American movies this year and only 650,000 last 

year. 

■ Ideal Performance: 125% increase. 

■ Minimally Acceptable Performance: 25% decrease. 

GENERAL POPULATON 
POLICY 

Favorability — General Population — Policy— Security (FGPPS) 
FGPPS Visa Applications 

■ Definition : The number of US visa applicants will show the desire of people to 
travel to the United States. 

■ Unit of Measure : Number time 100 

■ Calculation : The number of people who applied this year multiplied by 100 

■ Example : This year 27 people applied to travel from Nigeria to the US, multiplied 
by 100 is 270 

■ Ideal Performance : 100 applicants 

■ Minimally Acceptable Performance : 25 fewer than previous FY 
FGPPS Protests 

■ Definition : Number of public protests against US military activities 

■ Unit of Measure : Number count 

■ Calculation : The number of recorded public protests 
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■ Example : This year 3 public protests against US military activity were recorded 

■ Ideal Performance : 0 protests 

■ Minimally Acceptable Performance : 15 protests 

Favorabilitv — General Population — Policy — Economic (FGPPE) 
FGPPE Sale of US Goods 

■ Definition: Relative change in the sale of US goods and services 

■ Unit of Measure: Percent change 

■ Calculation: Percentage increase in US goods and services sold within that country 
while using year 2005 as a base year to deflate 

■ Example: US goods and services in India totaled $500,000,000 (in 2005 dollars) 
this year as opposed to $400,000,000 (in 2005 dollars) last year. (125% increase) 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 

FGPPE Job Application 

■ Definition: Relative change in the number of applicants wanting to work for the US 

■ Unit of Measure: Percent change 

■ Example: 100 applicants in 2009 and 200 applicants in 2010. 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 

FGPPE # US Franchises 

■ Definition: Number of new franchise licenses granted by US corporations to host- 
nation entities. 

■ Unit of Measure: Number count 
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■ Example: 117 Algerian entities were granted US franchise licenses in FY08 

■ Ideal Performance : 200 

■ Minimally Acceptable Performance : 0 

Favorability — General Population — Culture (FGPC) 
FGPC Exchange Appl'ns 

■ Definition: Relative change in the number of applicants from a country wanting to 
do an exchange program in the US from year to year 

■ Unit of Measure: Percent change 

■ Calculation: Percentage increase of applicants from a country wanting to do an 
exchange program in the US from year to year. 

■ Example: Last year Lebanon had 140 applicants while this year Lebanon has 250 
applicants wanting to participate in US exchange programs. (250 / 140 * 100 = 
178.5% increase) 

■ Ideal Performance : 50% increase 

■ Minimally Acceptable Performance : 25% decrease 

FGPC Survey Results 

■ Definition: Average survey rating of the US of all submitted ranks (1-10) indicated 
on the survey/poll issued at a US embassy social event 

■ Unit of Measure: Average rating of all submitted 

■ Calculation : standardization and summation of all the responses to make respective 
indices based on weights. 

■ Example: Last year Pakistan voted the US, on average, a 4. This year the average 
ranking was a 9. Favorability of the US is increasing among Pakistan's citizens. 

■ Ideal Performance :10 

■ Minimally Acceptable Performance : 1 
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FGPC Media Outlet Grwth 

■ Definition: Number of new US produced mass-market publications, (e.g. 
magazines, comic books, etc.) 

■ Unit of Measure: Number count 

■ Calculation : Count the number of new US produced mass-market publications, (e.g. 
magazines, comic books, etc.) 

■ Example: Last year Pakistan voted added 5 new US produced mass-market 
publications 

■ Ideal Performance : 20 

■ Minimally Acceptable Performance : 0 



106 



Section III: Influence 
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FOREIGN GOVERNMENT 
POLICY 

Influence — Foreign Government — Policy — Security (IFGPS) 
IFGPS Favorable Opinions 

■ Definition: Relative change of opinions of senior governmental officials after a 
relevant PD effort 

■ Unit of Measure: Percent change of opinion pre and post PD events 

■ Calculation: The number of senior government officials who changed their position 
over the total number of senior government officials attending PD events 

■ Example: A 30% increase of senior government official support of US request for 
cooperation in enforcing sanctions against Iran after attending PD events 

■ Ideal Performance : 75% increase 

■ Minimally Acceptable Performance : 0% increase 

IFGPS 3 rd Party Assistance 

■ Definition : Relative change in the number of times a foreign government agrees to 
assist in enforcing US preferred security issues with a third party state, after a 
relevant PD effort 

■ Unit of Measure : Percent change 

■ Calculation : The number of policy related events directed at members of foreign 
governments divided by the number of dedicated PD staff at the US Consulate 

■ Example : Percent change in (Number of policy info sessions/ PD staff) 

■ Ideal Performance : 20% increase 

■ Minimally Acceptable Performance : 25% decrease 

Influence — Foreign Government — Policy — Economy (TFGPE) 
IFGPE Trade Value 
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■ Definition: Relative change in the real value of a country's imports and exports with 
US in areas in a specific region or industry targeted by US PD efforts. 

■ Unit of Measure : Percent change 

■ Calculation : Difference between current year percentage of Trade value with US 
and previous FY 

■ Example 1 : An increase/decrease in a country's trade with the US over a five year 
period, as measured by one or more of the following: Absolute dollar value of 
imports/exports, percent of country's external trade with US, percent of a country's 
GDP representing trade with US 

■ Example 2 : Decrease in tariffs placed on US imports over a five year period 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 

IFGE Economic Forums 

■ Definition : Relative change in the number of economic forums where trade with 
US in encouraged 

■ Unit of Measure: Percent change 

■ Calculation : Compare the number of economic forums in 2008 to that of 2009 

■ Example: 1 economic forum in FY 2008, 1 forum in FY 2009 = 100% increase 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 

IFGPE Trade Agreements 

■ Definition: Number of new trade agreements established between the target nation 
and the US (or its trading partners) during the past year 

■ Unit of Measure: Number count 

■ Calculation: The number of new trade agreements established by the US with the 
target nation 



109 



■ Example: The US created a new trade agreement with Morocco and helped the E.U. 
to establish an FTA with Morocco in FY06, meaning 2 trade agreements 

■ Ideal Performance : 3 

■ Minimally Acceptable Performance : 0 

CULTURE 

Influence — Foreign Government — Culture (IFGC) 
IFGC Educ'n Exchange 

■ Definition: Relative change of the number of governments who officially promote 
and endorse US educational exchanges, initiatives and programs 

■ Unit of Measure: Percent change 

■ Calculation: The number of governments who changed their position regarding 
exchanges over the total number of governments who allowed exchanges the 
previous year 

■ Example: A 20% increase of governments allowing exchanges after a fiscal year 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 

IFGC Reading Materials 

■ Definition : Relative change in the number of materials that reach foreign 
audiences to detect increased political cooperation and receptiveness towards US 
educational materials 

■ Unit of Measure : Percent change 

■ Calculation : The number of materials in American-sponsored libraries and 
embassies over the number of materials in libraries and embassies in the prior FY 

■ Example : 20% increase in materials on the ground 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 
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ENVIRONMENT 

Influence — Foreign Government — Environment (IFGE) 



IFGE Agreements 

■ Definition : Relative change in the number of environmental agreements (multi- 
lateral and bilateral) signed by target country where PD efforts actively promoted 
support for agreements' subject matter 

■ Unit of Measure : Percent change 

■ Calculation : The number of signed agreements in current year divided by the 
number agreements signed during the previous year. 

■ Example : 0 agreements in FY 2008, 2 agreements in FY 2009 = co increase 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 



IFGE Initiatives 

■ Definitions : Relative change in number of new environmental initiatives 
implemented by the government. Examples of which would include national 
parks (or other protected areas), sustainable management of forests, protection of 
specific species, limits on air pollutants, etc. 

■ Unit of Measure: Percent change 

■ Calculation : The number of new environmental initiatives implemented in current 
year divided by the number of environmental initiatives enacted during the 
previous year. 

■ Example: 0 initiatives in FY 2008, 7 initiatives in FY 2009 = co increase 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 



ELITES 
POLICY 

Influence— Elites— Policy— Security (IEPS) 
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IEPS Afghanistan Opinion 

■ Definition: Relative change in the opinion of prominent elites after a PD event 
relating to US military presence in Afghanistan 

■ Unit of Measure: Percent change of opinion pre and post PD events 

■ Calculation: The number of elites who changed their position over the total 
number of elites who attending relevant PD events 

■ Example: A 30% increase of support of US military actions in Afghanistan among 
elites who attended relevant PD 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 

IEPS Exchange Growth 

■ Definition: Relative change from previous year in exchanges between host nation 
and US security agencies for purposes of training, development, and information 
sharing. 

■ Unit of Measure: Percent change 

■ Calculation: The number of security exchanges this FY minus those security 
exchanges of the previous FY, divided by the number from the previous FY 

■ Example: A 20% increase in security exchanges between the US and Poland in 
FY07 

■ Ideal Performance : 100% increase 

■ Minimally Acceptable Performance : 100% decrease 

Influence — Elites — Policy — Economic (IEPE) 
IEPE Bus. Partnerships 

■ Definition : The number of business partnerships involving 1 or more country elites 
in either a board member or executive capacity 

■ Unit of Measure: Number count 



112 



■ Calculation : Count in the number of joint ventures as compared with the previous 
5 or 10 year periods 

■ Example 1 : In 2000 there were 2 joint ventures between US companies and in- 
country firms involving 1 or more elites. In 2005, there were 10 such ventures 
representing a 500% increase. 

■ Ideal Performance : 10 partnership increase 

■ Minimally Acceptable Performance : 5 partnership decrease 

IEPE Work Visas 

■ Definition : Relative change in desired intervals of a country's citizen's work, 
business, and student visa requests to the US 

■ Unit of Measure : Percent change 

■ Calculation : The difference between the number of people who have requested 
visas in 2008 and that of 2009 

■ Example : 1000 request in FY 2008, 2000 in FY 2009 = 100% increase 

■ Ideal Performance : 50% increase 

■ Minimally Acceptable Performance : 10% decrease 

IEPE US Taxes Paid 

■ Definition : Amount of taxes paid to the US Government by host nation businesses 
in the past year for conducting business in the US 

■ Unit of Measure: Millions of Dollars 

■ Calculation : Total dollar amount (in millions) of taxes paid to US government by 
host nation businesses in a FY 

■ Example 1 : In 2000 Peruvian businesses paid $27 million in taxes to the US 
government 

■ Ideal Performance : $50 
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Minimally Acceptable Performance : $0 



CULTURE 

Influence — Elites — Culture (IEC) 
IEC Colloquia 

■ Definition: Relative change in the number of colloquiums and symposiums 
discussing the benefits of the US educational system 

■ Unit of Measure : Percent change 

■ Calculation: The number of elites who changed their position over the total 
number of elites who attended colloquiums divided by the previous year's figures. 

■ Example: A 35% increase of support for the US higher education system 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 

IEC Std. of Living Survey 

■ Definition : - Score of opinions concerning cultural and educational initiatives 
aimed at increasing the standard of life in their home country 

■ Unit of Measure : Score 

■ Calculation : The survey score of elites who are pleased with the direction that PD 
efforts 

■ Example : A score of 7 is equivalent to 70 on a 100 point scale 

■ Ideal Performance : 10 

■ Minimally Acceptable Performance : 0 

ENVIRONMENT 

Influence — Elites — Environment (IEE) 
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IEE Bus. Env'l Initiatives 

■ Definitions : Relative change in the number of target country businesses that support 
- through sponsorship of and/or participation in - PD efforts that focus on 
environmental initiatives. 

■ Unit of Measure: Percent change 

■ Calculation : The number of participating/sponsoring businesses per PD effort in 
current year divided by the number businesses participating in/sponsoring PD 
efforts during the previous year. 

■ Example : Average of 3.3 participating/sponsoring business per PD effort in FY 
2008, average of 6.5 participating/sponsoring business per PD effort in FY 2009 = 
97% increase 

■ Ideal Performance : 200% increase 

■ Minimally Acceptable Performance : 0% increase 

IEE Publicly Support 

■ Definitions : Relative change in the number of elites who publicly demonstrate 
through public statements and/or actions in support of environmental initiatives 
promoted by PD efforts. 

■ Unit of Measure : Percent change 

■ Calculation : The number of public statements and/or actions in current year 
divided by the number public statements and/or actions during the previous year. 

■ Example : 12 public statements and/or actions in FY 2008, 20 public statements 
and/or actions in FY 2009 = 67% increase 

■ Ideal Performance : 100% increase 

■ Minimally Acceptable Performance : 10% decrease 

GENERAL POPULATION 
POLICY 

Influence — General Population — Policy — Security (IGPPS) 



115 



IGPPS Afghanistan Opinion 

■ Definition: Score from "man-on-the-street" interviews concerning US 
involvement and or strategy in Afghanistan 

■ Unit of Measure: Score on a Likert scale 

■ Calculation: Score from interviews can be averaged 

■ Example: The average score given by Ethiopians regarding US strategy in 
Afghanistan in FY06 was a 4 

■ Ideal Performance : 10 

■ Minimally Acceptable Performance : 0 

IGPPS Media Wording 

■ Definition : - Relative change in the number of national security key words 
witnessed in privately controlled news media sources or blogs regarding such 
issues, as compared over time with relevant PD efforts targeted at elites 

■ Unit of Measure : - Percent change 

■ Calculation : The number of policy related events directed at the general public 
divided by the number of dedicated PD staff at the US Consulate 

■ Example : Percent change in (Number of policy info sessions/ PD staff) 

■ Ideal Performance : 125% increase 

■ Minimally Acceptable Performance : 25% decrease 

IGPPS Anti-U.S. Riots 

■ Definition: N umber of "violent" anti-U.S. riots by the general populace anywhere 
within the host nation in the past year 

■ Unit of Measure: Number count 

■ Calculation: The number of "violent" anti-U.S. riots within the host-country's 
borders 
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■ Example: Tajikistan had 3 "violent" anti-U.S. riots in FY07 

■ Ideal Performance : 0 

■ Minimally Acceptable Performance : 5 

Influence — General Population — Policy — Economic (IGPPE) 
IGPPE Job Creation 

■ Measurement : Relative change per desired intervals of a country's number of jobs 
created in a sector with US PD involvement 

■ Unit of Measure: Percent change 

■ Calculation : Subtract the percentage of previous year's jobs from current year's 
jobs. 

■ Example : 1000 jobs created in 2008 in a sector with US PD involvement, 2000 jobs 
created in 2009. 100% increase 

■ Ideal Performance: 125% increase. 

■ Minimally Acceptable Performance: 25% decrease. 

CULTURE 

Influence — General Population — Culture (IGPC) 
IGPC Spt Exchange Increasing 

■ Definition: Relative change in opinion polls of students who have participated in 
educational exchanges with the United States which state their level of satisfaction 
with their experience 

■ Unit of Measure : Percent change 

■ Calculation: The number of people who support increasing the number of students 
who attend American schools divided by those who supported increasing the 
number of students in the previous fiscal year 

■ Example: A 30% increase in public support for increasing bilateral student 
exchanges. 
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■ Ideal Performance : 100% increase 

■ Minimally Acceptable Performance : 0% increase 

IGPC Literacy Rates 

■ Definition : Relative change in literacy rates in countries where the United States 
has pushed public literacy programs 

■ Unit of Measure: Percent change 

■ Calculation : Demographically separated literacy rates divided by identically 
demographically separated literacy rates from the previous year. 

■ Example : A 15% increase in women's literacy in Afghanistan. 

■ Ideal Performance : 100% increase 

■ Minimally Acceptable Performance : 10% increase 

IGPC Box Office Gross 

■ Definition : Gross receipts for tickets to US produced films and movies presented in 
the host nation in the past year 

■ Unit of Measure: Thousands of Dollars 

■ Calculation : Total dollar amount (in thousands) of gross receipts for tickets to US 
government by host nation businesses in a FY 

■ Example 1 : In 2008 gross receipts for US movies in Colombia was $27 thousand 

■ Ideal Performance : $250 

■ Minimally Acceptable Performance : $50 

ENVIRONMENT 

Influence — General Population — Environment (IGPE) 
IGPE Pub's Opinion Survey 
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■ Definition : Score of US policies with regard to the environment from participants 
involved in a PD initiative/event 

■ Unit of Measure: Score on a Likert scale 

■ Calculation : Average participants' scores 

■ Example : The average score for US policies with regard to the environment was 7 
following a PD event in Sweden 

■ Ideal Performance : 10 

■ Minimally Acceptable Performance : 0 

IGPE Recognition Poll 

■ Definition : Relative change in the number of people who recognize environmental 
initiatives as important to the progress of their country 

■ Unit of Measure: Percent change 

■ Calculation : Subtract the percentage of previous year's poll respondents from 
current year's poll respondents. 

■ Example : 40% agree that environmental initiatives are important during previous 
year and 65% agree this year = 25% improvement or 62.5% increase. 

■ Ideal Performance: 100% increase. 

■ Minimally Acceptable Performance : 0% increase 
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Appendix D: Public Diplomacy Performance Assessment 
Model with Example Data 



This appendix contains a sample implementation of the Public Diplomacy - Model for 
Assessing Performance (PD-MAP). This sample is presented in three parts: 



1 . A brief "Users Guide" for entering the information required for the model to 
function. This guide, "Using PD-MAP", explains the type of information to be 
entered for each of the model elements. 



2. A hierarchical diagram of the example PD-MAP using the outcomes, audiences, 
policy areas, sub-outcomes, and metrics described in the report. Note that model 
elements shown in bold in this model, include the example metrics described in 
Appendix C. Unbolded metrics are purely "notional" and/or fictional, for 
demonstrating the model. Standards of performance, priorities, and risk values 
portrayed in this model are also notional, for purposes of demonstrating the 
model. Users are invited to develop their own metrics, priorities, etc. 

3. A collection of the graphical output for the "Understanding" Outcome of the 
example PD-MAP described in this appendix. 



120 



Using PD-MAP 



The steps below provide a guide to setting up and using the Public Diplomacy-Model 
for Assessment of Programs (PD-MAP). These steps are intended to be guidelines 
for completing the various input elements of the model. PD-MAP users are 
encouraged to develop their own outcomes, performance metrics, etc. 

PD-MAP is a hierarchical spreadsheet model that uses the concept of "weighted 
utility" scoring (from Multi-Criteria Decision theory) to produce a performance 
"score" on a 0-100 scale, against a desired level of performance. The hierarchical 
structure of the model is fixed in its current configuration, but the spreadsheet 
implementation provides the means for expanding the model in terms of outcomes 
(model breadth) and metrics (model depth). 

When the model input elements have been entered, PD-MAP automatically generates 
a series of bar graphs to represent the cumulative weighted performance score at each 
"level" of the model. A table of numerical performance scores is also generated for 
reference, corresponding to each bar graph. Thus, overall relative performance, on a 
scale of 0-100 can be broken down into its component performance scores at every 
level of the model. This enables the user to identify relatively strong and weak 
performance throughout the model. 

Building the PD-MAP 

1. Determine the appropriate 'viewpoint' for the analysis. A committee of 
stakeholders is a viable body to determine the viewpoint. Those with a vested 
interest in the outcome of the decision would be public affairs officers, heads of 
public affairs sections and the Undersecretary for Public Diplomacy and Public 
Affairs, for example. This is particularly useful in the strategic planning process. 
Determining the stakeholders enables the appropriate level and model purpose to 
be developed. 

2. Identify the elements for evaluation. For purposes of the example presented in this 
report, these elements include: 

o The audience - At whom will the PD efforts be "targeted" or intended to 
reach? For this particular example, the project team selected the audiences of 
foreign governments, elites, and the general population, as explained in the report. 

o Outcomes - Represent to anticipated ultimate effect of the PD effort(s). As 
developed in the report, this example uses the outcomes of "Understanding", 
"Favorability", and "Influence". This list is by no means exhaustive and other 
outcomes can be added. 
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o The policy area(s) - The "targeted" policy areas presented in this example were 
outlined in the report and include: US Foreign Policy, US Economic Policy, US 
Security Policy, US Environmental Policy, and US Culture. 

o Programs and/or efforts that lead to the outcomes - As presented in the 
"Understanding" outcome in this model, these "efforts" might be considered 
"sub-outcomes". The Understanding outcome includes the sub-outcomes of: 
Dissemination, Reception, and Comprehension. These terms were also explained 
in Chapter Four of the report. 

Establish the priority for each level of analysis. Priority should be assigned to the 
outcome, the audience, the policy area, the sub-outcome and finally, the effort or 
activity. Priority is indicated on a 0 to 10 scale, where zero (0) indicates the item 
is of absolutely no importance or relevance to the PD effort, and a ten (10) 
indicates the highest priority. (Note that not every effort in public diplomacy can 
have a priority of 10.) If data is missing or an audience is not involved in a 
particular effort, the priority is listed as zero. 

Identify the variables to be measured (metrics) for each program/effort/sub- 
outcome. These metrics are the quantifiable and measurable items that are used to 
define a successful result. These metrics often take the form of output measures. 
For each quantifiable "metric" in the model, the user establishes an expected 
"ideal" result and a minimally acceptable result, as well as the units of measure 
for the result, (e.g. Likert Scale, percent, number of occurrences, etc.). These two 
standards (ideal and minimum) establish the bounds of the scoring scale for 
determining the results of the PD efforts, where a score of zero (0) represents the 
minimally acceptable performance and a score of 100 represents the ideal 
performance. For each metric, the user also establishes a relative priority for 
achieving the desired result, as was done for the outcomes, etc., in step 3 above. 

Identify the risk associated with attaining the desired performance for each metric. 
Risk refers to the willingness to commit resources relative to the expected output. 
The model provides for categorizing three risk behaviors: 

o Risk Averse - The decision maker is averse to committing significant resources 
to attain the highest level of expected performance and tends to be more satisfied 
with incremental gain at the lower end of the performance scale. 

o Risk Seeker - The decision maker is willing to commit significant resources to 
attain the highest level of expected performance. This decision maker is less 
concerned about achieving low performance and is more willing to do what it 
takes to achieve the highest performance levels. 

o Risk Neutral - Often considered the "expected value" decision maker, this 
person's attitude about committing resources or efforts to attaining a desired level 
of performance does not change, regardless of the expected or desired level of 
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performance. They are considered "neutral" when it comes to taking risks, (i.e., 
neither willing to take a risk, nor afraid to take a risk). 

In this model, risk is depicted on a scale of -10 to 10. If the decision maker is 
considered a risk taker, this is indicated by a positive value of the risk number. If 
a decision maker is considered a risk avoider, then a negative number is used to 
depict this persons risk attitude for that particular metric. Risk neutral is 
represented by "0". The further from zero the risk value, the more "extreme" is 
attitude toward risk with respect to that particular metric. Note that each metric in 
the model is assigned a "risk value" to represent the decision makers risk attitude. 
Each metric could be measured by a different risk value. For example, a decision 
maker might be very willing to take significant risks (political or otherwise) to 
avoid anti-US demonstrations occurring in a country, but not so willing to take 
significant risks to expand the US visa acceptance rate for a country. 

6. Observe the results. The graphical output of this model allows the user to compare 
the performance of multiple outcomes, sub-outcomes, programs and other 
components to each other over multiple periods of time. The overall performance 
score represented for each element in the model is depicted as a sum of its 
component measures. This provides the user the capability to "trace" an outcome 
result back to its "cause". 

7. Cost-effectiveness, "what-if ', and sensitivity analysis can be performed by 
manipulating the risk, priority, ideal and minimally acceptable values in the 
model. This provides the decision maker(s) with a range of possible outcomes, 
based on historical results. This is useful in planning future PD efforts and 
expectations. Cost-effectiveness can be measured in terms of "bang-for-the- 
buck" simply by dividing the performance "score" by the cost of achieving that 
level of performance. 
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